jepler/circuitpython-build-tools
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Initial version. Still subject to change.

Browse source
This commit is contained in:
Scott Shawcroft 2017-11-16 18:47:11 -08:00
commit 9301c956a4
9 changed files with 534 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored Normal file
View file

@ -0,0 +1 @@
__pycache__

74
CODE_OF_CONDUCT.md Normal file
View file

@ -0,0 +1,74 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
nationality, personal appearance, race, religion, or sexual identity and
orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at support@adafruit.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at [http://contributor-covenant.org/version/1/4][version]
[homepage]: http://contributor-covenant.org
[version]: http://contributor-covenant.org/version/1/4/

21
LICENSE Normal file
View file

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2017 Scott Shawcroft for Adafruit Industries
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

7
README.md Normal file
View file

@ -0,0 +1,7 @@
# Adafruit CircuitPython Build Tools
[![Discord](https://img.shields.io/discord/327254708534116352.svg)](https://discord.gg/nBQh6qu)
This repo contains build scripts used to build the
[Adafruit CircuitPython bundle](https://github.com/adafruit/Adafruit_CircuitPython_Bundle), [CircuitPython Community bundle](https://github.com/adafruit/CircuitPython_Community_Bundle)
and individual library release zips.

117
circuitpython_build_tools.py Normal file
View file

@ -0,0 +1,117 @@
#!/usr/bin/env python3
# The MIT License (MIT)
#
# Copyright (c) 2016 Scott Shawcroft for Adafruit Industries
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import os
import os.path
import shutil
import sys
import subprocess
IGNORE_PY = ["setup.py", "conf.py"]
def build_mpy_cross(mpy_cross_filename, circuitpython_tag, quiet=False):
if not quiet:
title = "Building mpy-cross for circuitpython " + circuitpython_tag
print()
print(title)
print("=" * len(title))
os.makedirs("build_deps/", exist_ok=True)
if not os.path.isdir("build_deps/circuitpython"):
clone = subprocess.run("git clone https://github.com/adafruit/circuitpython.git build_deps/circuitpython", shell=True)
if clone.returncode != 0:
sys.exit(clone.returncode)
current_dir = os.getcwd()
os.chdir("build_deps/circuitpython/mpy-cross")
make = subprocess.run("git fetch && git checkout {TAG} && git submodule update && make clean && make".format(TAG=circuitpython_tag), shell=True)
os.chdir(current_dir)
shutil.copy("build_deps/circuitpython/mpy-cross/mpy-cross", mpy_cross_filename)
if make.returncode != 0:
sys.exit(make.returncode)
def build_library(library_path, output_directory, mpy_cross=None):
py_files = []
package_files = []
total_size = 512
for filename in os.listdir(library_path):
full_path = os.path.join(library_path, filename)
init_file = os.path.join(full_path, "__init__.py")
if os.path.isdir(full_path) and os.path.isfile(init_file):
files = os.listdir(full_path)
files = filter(lambda x: x.endswith(".py"), files)
files = map(lambda x: os.path.join(filename, x), files)
package_files.extend(files)
if filename.endswith(".py") and filename not in IGNORE_PY:
py_files.append(filename)
if len(py_files) > 1:
output_directory = os.path.join(output_directory, library)
os.makedirs(output_directory)
package_init = os.path.join(output_directory, "__init__.py")
# Touch the __init__ file.
with open(package_init, 'a'):
pass
if len(package_files) > 1:
for fn in package_files:
base_dir = os.path.join(output_directory, os.path.dirname(fn))
if not os.path.isdir(base_dir):
os.makedirs(base_dir)
total_size += 512
new_extension = ".py"
if mpy_cross:
new_extension = ".mpy"
for filename in py_files:
full_path = os.path.join(library_path, filename)
output_file = os.path.join(output_directory,
filename.replace(".py", new_extension))
if mpy_cross:
mpy_success = subprocess.call([mpy_cross,
"-o", output_file,
full_path])
if mpy_success != 0:
raise RuntimeError("mpy-cross failed on", full_path)
else:
shutil.copyfile(full_path, output_file)
for filename in package_files:
full_path = os.path.join(library_path, filename)
output_file = os.path.join(output_directory,
filename.replace(".py", new_extension))
if (not mpy_cross or
os.stat(full_path).st_size == 0 or
filename.endswith("__init__.py")):
shutil.copyfile(full_path, output_file)
else:
mpy_success = subprocess.call([mpy_cross,
"-o", output_file,
full_path])
if mpy_success != 0:
raise RuntimeError("mpy-cross failed on", full_path)

132
scripts/build_bundles.py Executable file
View file

@ -0,0 +1,132 @@
#!/usr/bin/env python3
# The MIT License (MIT)
#
# Copyright (c) 2016 Scott Shawcroft for Adafruit Industries
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import os
import os.path
import shlex
import shutil
import sys
import subprocess
import zipfile
import circuitpython_build_tools
def add_file(bundle, src_file, zip_name):
bundle.write(src_file, zip_name)
file_size = os.stat(src_file).st_size
file_sector_size = file_size
if file_size % 512 != 0:
file_sector_size = (file_size // 512 + 1) * 512
print(zip_name, file_size, file_sector_size)
return file_sector_size
def build_bundle(lib_location, bundle_version, output_filename,
mpy_cross=None):
build_dir = "build-" + output_filename
build_lib_dir = os.path.join(build_dir, "lib")
if os.path.isdir(build_dir):
print("Deleting existing build.")
shutil.rmtree(build_dir)
os.mkdir(build_dir)
os.mkdir(build_lib_dir)
success = True
total_size = 512
for subdirectory in os.listdir("libraries"):
for library in os.listdir(os.path.join("libraries", subdirectory)):
library_path = os.path.join("libraries", subdirectory, library)
circuitpython_build_tools.build_library(library_path,
build_lib_dir,
mpy_cross=mpy_cross)
with open(os.path.join(build_lib_dir, "VERSIONS.txt"), "w") as f:
f.write(bundle_version + "\r\n")
versions = subprocess.run('git submodule foreach \"git remote get-url origin && git describe --tags\"', shell=True, stdout=subprocess.PIPE)
repo = None
for line in versions.stdout.split(b"\n"):
if line.startswith(b"Entering") or not line:
continue
if line.startswith(b"git@"):
repo = b"https://github.com/" + line.split(b":")[1][:-len(".git")]
elif line.startswith(b"https:"):
repo = line.strip()[:-len(".git")]
else:
f.write(repo.decode("utf-8", "strict") + "/releases/tag/" + line.strip().decode("utf-8", "strict") + "\r\n")
with zipfile.ZipFile(output_filename, 'w') as bundle:
total_size += add_file(bundle, "README.txt", "lib/README.txt")
for filename in os.listdir("update_scripts"):
src_file = os.path.join("update_scripts", filename)
total_size += add_file(bundle, src_file, os.path.join("lib", filename))
for root, dirs, files in os.walk(build_lib_dir):
ziproot = root[len(build_dir + "/"):].replace("-", "_")
for filename in files:
total_size += add_file(bundle, os.path.join(root, filename),
os.path.join(ziproot, filename.replace("-", "_")))
print()
print(total_size, "B", total_size / 1024, "kiB", total_size / 1024 / 1024, "MiB")
print("Bundled in", zip_filename)
if not success:
print("WARNING: some failures above")
sys.exit(2)
if __name__ == "__main__":
from scripts import target_versions
tagged = input("Did you tag this release already ([y]/n)? ")
if tagged and tagged.lower() != 'y':
print("Go ahead and tag. I'll wait.")
sys.exit(3)
bundle_lib_location = os.path.abspath(sys.argv[1])
output_dir = os.path.abspath(sys.argv[2])
os.chdir(bundle_lib_location)
bundle_version = None
tag = subprocess.run('git describe --tags --exact-match', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if tag.returncode == 0:
bundle_version = tag
else:
commitish = subprocess.run("git log --pretty=format:'%h' -n 1", shell=True, stdout=subprocess.PIPE)
bundle_version = commitish
bundle_version = bundle_version.stdout.strip().decode("utf-8", "strict")
zip_filename = os.path.join(output_dir,
'adafruit-circuitpython-bundle-{VERSION}.zip'.format(
VERSION=bundle_version))
build_bundle(bundle_lib_location, bundle_version, zip_filename)
os.makedirs("build_deps", exist_ok=True)
for version in target_versions.VERSIONS:
mpy_cross = "build_deps/mpy-cross-" + version["name"]
circuitpython_build_tools.build_mpy_cross(mpy_cross, version["tag"])
zip_filename = os.path.join(output_dir,
'adafruit-circuitpython-bundle-{TAG}-{VERSION}.zip'.format(
TAG=version["name"],
VERSION=bundle_version))
build_bundle(bundle_lib_location, bundle_version, zip_filename, mpy_cross=mpy_cross)

123
scripts/build_library.py Normal file
View file

@ -0,0 +1,123 @@
#!/usr/bin/env python3
# The MIT License (MIT)
#
# Copyright (c) 2016 Scott Shawcroft for Adafruit Industries
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import os
import os.path
import shlex
import shutil
import sys
import subprocess
import zipfile
import circuitpython_build_tools
def add_file(bundle, src_file, zip_name):
bundle.write(src_file, zip_name)
file_size = os.stat(src_file).st_size
file_sector_size = file_size
if file_size % 512 != 0:
file_sector_size = (file_size // 512 + 1) * 512
print(zip_name, file_size, file_sector_size)
return file_sector_size
def build_bundle(lib_location, bundle_version, output_filename,
mpy_cross=None):
build_dir += "-" + output_filename
build_lib_dir = os.path.join(build_dir, "lib")
if os.path.isdir(build_dir):
print("Deleting existing build.")
shutil.rmtree(build_dir)
os.mkdir(build_dir)
os.mkdir(build_lib_dir)
success = True
total_size = 512
for subdirectory in os.listdir("libraries"):
for library in os.listdir(os.path.join("libraries", subdirectory)):
library_path = os.path.join("libraries", subdirectory, library)
circuitpython_build_tools.build_library(library_path,
build_lib_dir,
mpy_cross=mpy_cross)
with open(os.path.join(build_lib_dir, "VERSIONS.txt"), "w") as f:
f.write(bundle_version + "\r\n")
versions = subprocess.run('git submodule foreach \"git remote get-url origin && git describe --tags\"', shell=True, stdout=subprocess.PIPE)
repo = None
for line in versions.stdout.split(b"\n"):
if line.startswith(b"Entering") or not line:
continue
if line.startswith(b"git@"):
repo = b"https://github.com/" + line.split(b":")[1][:-len(".git")]
elif line.startswith(b"https:"):
repo = line.strip()[:-len(".git")]
else:
f.write(repo.decode("utf-8", "strict") + "/releases/tag/" + line.strip().decode("utf-8", "strict") + "\r\n")
with zipfile.ZipFile(output_filename, 'w') as bundle:
total_size += add_file(bundle, "README.txt", "lib/README.txt")
for filename in os.listdir("update_scripts"):
src_file = os.path.join("update_scripts", filename)
total_size += add_file(bundle, src_file, os.path.join("lib", filename))
for root, dirs, files in os.walk(build_lib_dir):
ziproot = root[len(build_dir + "/"):].replace("-", "_")
for filename in files:
total_size += add_file(bundle, os.path.join(root, filename),
os.path.join(ziproot, filename.replace("-", "_")))
print()
print(total_size, "B", total_size / 1024, "kiB", total_size / 1024 / 1024, "MiB")
print("Bundled in", zip_filename)
if not success:
print("WARNING: some failures above")
sys.exit(2)
if __name__ == "__main__":
tagged = input("Did you tag this release already ([y]/n)? ")
if tagged and tagged.lower() != 'y':
print("Go ahead and tag. I'll wait.")
sys.exit(3)
version = None
tag = subprocess.run('git describe --tags --exact-match', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if tag.returncode == 0:
version = tag
else:
commitish = subprocess.run("git log --pretty=format:'%h' -n 1", shell=True, stdout=subprocess.PIPE)
version = commitish
version = version.stdout.strip().decode("utf-8", "strict")
zip_filename = os.path.join(build_dir,
'adafruit-circuitpython-bundle-{TAG}-{VERSION}.zip'.format(TAG=circuitpython_tag,
VERSION=version))
bundle_lib_location = sys.argv[1]
build_bundle(build_lib_location, output_filename)
for version in target_versions.VERSIONS:
if os.path
build_bundle(version["name"], mpy_cross="build_deps/mpy-cross-" + version["name"])

30
scripts/build_mpy_cross.py Normal file
View file

@ -0,0 +1,30 @@
#!/usr/bin/env python3
# The MIT License (MIT)
#
# Copyright (c) 2016 Scott Shawcroft for Adafruit Industries
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import circuitpython_build_tools
if __name__ == "__main__":
output_filename = sys.argv[1]
circuitpython_tag = sys.argv[2]
circuitpython_build_tools.build_mpy_cross(output_filename, circuipython_tag)

29
scripts/target_versions.py Normal file
View file

@ -0,0 +1,29 @@
#!/usr/bin/env python3
# The MIT License (MIT)
#
# Copyright (c) 2016 Scott Shawcroft for Adafruit Industries
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
VERSIONS = [
{"tag": "1.0.0", "name": "1.x"},
{"tag": "2.0.0", "name": "2.x"},
{"tag": "3.0.0-alpha.1", "name": "3.x-beta"},
]