Merge pull request #218 from adafruit/BlitzCityDIY-patch-1

add fruit jam

.github/workflows/githubci.yml

@@ -1,20 +1,19 @@
 name: Github Arduino Library CI
 
-on: [push]
+on: [push, pull_request]
 
 jobs:
   build:
-
     runs-on: ubuntu-latest
     
     steps:
-    - uses: actions/checkout@v1
+    - uses: actions/checkout@v4
-    - uses: actions/setup-python@v1
+    - uses: actions/setup-python@v5
       with:
         python-version: '3.x'
     - name: pre-install
       run: bash ./actions_install.sh
     - name: test platforms
       run: |
-        python3 build_platform.py uno leonardo mega2560 zero esp8266 esp32 pico_rp2040
+        python3 build_platform.py uno leonardo mega2560 zero esp8266 esp32 pico_rp2040 feather_m4_express feather_rp2350
 

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+.idea

README.md

@@ -1,256 +1,93 @@
-# Travis CI Arduino Init Script [![Build Status](https://travis-ci.com/adafruit/travis-ci-arduino.svg?branch=master)](https://travis-ci.com/adafruit/travis-ci-arduino)
+# Arduino CI Scripts
 
-The purpose of this repo is to create a centrally managed dependency
+This repo contains various scripts and tools related to running continuous integration (CI) checks on Arduino Library Repos. The operations include:
-install script for all Adafruit Arduino Library Travis CI and Github Actions configs.
-This will allow us to easily update the install steps without
-having to manually or programmatically update 100+ `.travis.yml` files.
 
-We have a guide that you can use to follow along to install both TravisCI and Doxygen generation here https://learn.adafruit.com/the-well-automated-arduino-library/
+* checking formatting using [clang-format](https://clang.llvm.org/docs/ClangFormat.html),
+* generating documentation from source comments using [Doxygen](https://www.doxygen.nl/), and
+* building each example in the library for selected targets.
 
-## Adding to Travis CI Configs
+There is an associated guide available here:
+https://learn.adafruit.com/the-well-automated-arduino-library/
 
-You will need to source the script in the `before_install` step of your
+## Adding GitHub Actions to Repo
-`.travis.yml` file.
 
-```sh
+To run these continuous integration checks on each push, pull-request or [repository dispatch](https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#create-a-repository-dispatch-event) using [GitHub actions](https://github.com/features/actions):  
-source <(curl -SLs https://raw.githubusercontent.com/adafruit/travis-ci-arduino/master/install.sh)
+
+* Create a folder named `.github/worflows` in the root of the repo.
+* Copy `example_actions.yml` into the above directory and rename it `githubci.yml`.
+* Edit `githubci.yml` and change `PRETTYNAME` to the library repo name. Optionally, delete or comment out steps (using the `#` character), you don't want to include. 
+* Here's an example: [Adafruit_BME280_Library](https://github.com/adafruit/Adafruit_BME280_Library/blob/master/.github/workflows/githubci.yml)
+
+## Controlling Test Behavior
+
+The `build_platform.py` script is used to test each `.ino` example in the repo for selected build platforms. The [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54) dictionary contains a listing of all available platforms and selected platform groups. By default, `main_platforms` is used. To select a specific platform or group, replace `main_platforms` in [`githubci.yml`](`example_actions.yml`) with the group or platform name.
+
+Additionally, [UF2 files](https://github.com/microsoft/uf2) of the compiled sketches can be generated for supported platforms. 
+
+### Fine tuning test selection
+
+The script behavior can be controlled using special filenames: 
+
+* `.PLATFORM_ID.test.skip` - Skip the specified platform. All others are tested.
+* `.PLATFORM_ID.test.only` - Test the specified platform. All others are skipped.
+* `.PLATFORM_ID.generate` - Generate UF2 of sketch for specified platform (if supported).
+
+These are just empty files placed in an example folder. Replace `PLATFORM_ID` in the name with the key from [`ALL_PLATFORMS`](ci-arduino/blob/master/build_platform.py#L54). `metro_m0` from the following line in `build_platform.py`, for example:
+
+```python
+"metro_m0" : ["adafruit:samd:adafruit_metro_m0", "0x68ed2b88", None],
 ```
 
-If you only want to install and build on certain platforms, you can set the
+You can use several `.PLATFORM_ID.test.skip` or `.PLATFORM_ID.test.only` to exclude or include multiple platforms. For example:
-`INSTALL_PLATFORMS` environment variable to a comma-separated list of platforms.
+
+* To **skip** testing on ESP8266, add a file named `.esp8266.test.skip`
+* To test **only** the Arduino UNO, add a file named `.uno.test.only`
+* To skip all and test **nothing**, add a file named `.none.test.only`
+* To generate UF2s for PyPortal, add a file named `.pyportal.generate`
+
+### Dependencies
+
+Any library dependencies included in the [`library.properties`](https://arduino.github.io/arduino-cli/0.19/library-specification/#libraryproperties-file-format) are automatically installed before the tests are started. To install additional dependencies (e.g., those required for some examples but not the library itself) using [`arduino-cli`](https://arduino.github.io/arduino-cli/0.19/commands/arduino-cli_lib_install/), you could add additional steps to the `githubci.yml` file. For example:
 
-**Example `.travis.yml`:**
 ```yaml
-language: c
+- name: Set configuration
-sudo: false
+  run: arduino-cli config set library.enable_unsafe_install true 
-cache:
+
-  directories:
+- name: Install test dependencies
-    - ~/arduino_ide
+  run: arduino-cli lib install --git-url https://github.com/arduino-libraries/Servo --git-url https://github.com/arduino-libraries/Ethernet
-    - ~/.arduino15/packages/
+```
-git:
+
-  depth: false
+Note: you'll only need to enable the [`enable_unsafe_install`](https://arduino.github.io/arduino-cli/0.32/configuration/#configuration-keys) option if you want to identify libraries using urls. This isn't necessary when using the library name. 
-  quiet: true
+
-addons:
+## Formatting Check with Clang
-  apt:
+
-    sources:
+The `run-clang-format.py` script is used to run [clang-format](https://clang.llvm.org/docs/ClangFormat.html) and check file formatting.
-      - llvm-toolchain-trusty-5.0
+See [the guide](https://learn.adafruit.com/the-well-automated-arduino-library/formatting-with-clang-format) for details on installing `clang-format` to run formatting locally.
-      - key_url: 'http://apt.llvm.org/llvm-snapshot.gpg.key'
+Even a single extra white space can cause the CI to fail on formatting.
-    packages:
+You can typically just let clang do its thing and edit files in place using:
-      - clang-format-5.0
+
+```
+clang-format -i File_To_Format.cpp
+```
+
+## Documentation with Doxygen
+
+The `doxy_gen_and_deploy.sh` script uses [Doxygen](https://www.doxygen.nl/) to generate and deploy documentation
+for the library. Any issues, like missing documentation, will cause the CI to fail.
+See the [guide](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen) for details on installing and running Doxygen locally. The guide also has some
+[tips](https://learn.adafruit.com/the-well-automated-arduino-library/doxygen-tips) on basic usage of Doxygen markup within your code.
+
+### Preserving Folders in Documentation Branch
+
+By default, the documentation deployment script cleans the gh-pages branch before adding new documentation. If you need to preserve certain folders (like custom web interfaces), you can set the `PRESERVE_FOLDERS` environment variable in your workflow:
+
+```yaml
+- name: doxygen
   env:
-  global:
+    GH_REPO_TOKEN: ${{ secrets.GH_REPO_TOKEN }}
-#    - ARDUINO_IDE_VERSION="1.8.10"
+    PRETTYNAME : "My Arduino Library"
-     - PRETTYNAME="Adafruit FT6206 Arduino Library"
+    PRESERVE_FOLDERS: "webserial,assets"
-# Optional, will default to "$TRAVIS_BUILD_DIR/Doxyfile"
+  run: bash ci/doxy_gen_and_deploy.sh
-#    - DOXYFILE: $TRAVIS_BUILD_DIR/Doxyfile
-
-before_install:
-   - source <(curl -SLs https://raw.githubusercontent.com/adafruit/travis-ci-arduino/master/install.sh)
-   - curl -SLs https://raw.githubusercontent.com/adafruit/travis-ci-arduino/master/run-clang-format.py > run-clang-format.py
-
-install:
-   - arduino --install-library "Adafruit ILI9341","Adafruit GFX Library"
-
-script:
-   - python run-clang-format.py -r .
-   - build_main_platforms
-
-# Generate and deploy documentation
-after_success:
-  - source <(curl -SLs  https://raw.githubusercontent.com/adafruit/travis-ci-arduino/master/library_check.sh)
-  - source <(curl -SLs  https://raw.githubusercontent.com/adafruit/travis-ci-arduino/master/doxy_gen_and_deploy.sh)
 ```
 
-**Choosing Arduino IDE version**
+This will preserve the listed folders (comma-separated) during the documentation generation process.
-
-You could use any version of IDE by setting `ARDUINO_IDE_VERSION` variable but we recommend keeping this variable unused because script gets updated and you then will not have to modify `.travis.yml` manually.
-
-## Automated Example Verification Bash Functions
-
-`build_platform` will build all `.ino` examples in the repo using the passed platform. The platforms
-are defined in the `MAIN_PLATFORMS` and `AUX_PLATFORMS` associative arrays at the top of the script.
-
-All of the examples will be built with the platforms in `MAIN_PLATFORMS` if you call `build_main_platforms`,
-and `AUX_PLATFORMS` can be used to define other platforms that don't need to be verified for every repo.
-
-Build the examples using the platforms in the MAIN_PLATFORMS array:
-```yaml
-script:
-  - build_main_platforms
-```
-
-Build the examples only using the trinket:
-```yaml
-script:
-  - build_platform trinket
-```
-
-### Skipping Platforms
-
-If you would like to skip one of the main platforms when running `build_main_platforms`,
-you can commit a `.YOUR_PLATFORM_HERE.test.skip` file to the example sketch directory you
-wish to skip. You will need to use the array key defined in `MAIN_PLATFORMS` for the platform
-you wish to skip.
-
-For example, if you would like to skip the `esp8266` platform for an example
-in your lib called `blink.ino`, you would need to do something like this in your library repo:
-
-```sh
-$ touch examples/blink/.esp8266.test.skip
-$ git add -A
-$ git commit -a
-$ git push
-```
-
-If you need an easy way to skip a platform, you can also add something like this to your `~/.bash_profile`:
-
-```sh
-function travis_skip()
-{
-
-  local platform_key=$1
-
-  # grab all pde and ino example sketches
-  local examples=$(find $PWD -name "*.pde" -o -name "*.ino")
-
-  # loop through example sketches
-  for example in $examples; do
-
-    # store the full path to the example's sketch directory
-    local example_dir=$(dirname $example)
-
-    touch ${example_dir}/.${platform_key}.test.skip
-
-  done
-
-}
-```
-
-You will then be able to skip a platform for all examples by running the `travis_skip` function from your library repo.
-It will automatically add the `.YOUR_PLATFORM_HERE.test.skip` files to the examples.
-
-```sh
-$ travis_skip esp8266
-```
-
-## Using external libraries
-External libraries (which are not hosted by the Arduino library manager) can be installed using the following command:
-```sh
-- if [ ! -d "$HOME/arduino_ide/libraries/<Name>" ]; then git clone <URL> $HOME/arduino_ide/libraries/<Name>; fi
-```
-
-## Deploying compiled artifacts
-If you need to get hold of the compiled sketches of your project, in order to release them or forward them to an
-deployment pipeline, you can find them in the `$ARDUINO_HEX_DIR` directory. Specifically, if `Foo` is the name
-of your project, you are compiling for an `Arduino Mega` and the primary sketch is called `Foo.ino`, the flashable
-`.hex` files will be found inside `$ARDUINO_HEX_DIR/mega2560/Foo` as `Foo.ino.hex` and `Foo.ino.with_bootloader.hex`.
-Similarly for the rest of the platforms.
-
-For example, assuming you have a `Foo` project as outlined above, to create a release which includes the `.hex`
-files on GitHub, you could add this to your `.travis.yml` configuration:
-
-```yaml
-deploy:
-  provider: releases
-  api_key:
-    secure: YOUR_API_KEY_ENCRYPTED
-  file:
-    - $ARDUINO_HEX_DIR/mega2560/Foo/Foo.ino.hex
-    - $ARDUINO_HEX_DIR/mega2560/Foo/Foo.ino.with_bootloader.hex
-  skip_cleanup: true
-  on:
-    tags: true
-```
-
-## Running build_platforms.py locally
-1. Install arduino-cli from here: https://arduino.github.io/arduino-cli/installation/
-2. Download ci-arduino
-   * `git clone https://github.com/adafruit/ci-arduino`
-3. Put these lines at the end of your `.bashrc` or `.bash_profile` if you're on OSX. Make sure to fill in the path to where you installed ci-arduino and replacing USER with your username.
-   ```bash
-   alias test-platforms='python3 ~/path/to/ci-arduino/build_platform.py'
-   export HOME=/home/USER/
-   ```
-4. Run this at the top level of the library you want to test
-   ```bash
-   adafruit@adafruit:~/Adafruit_BMP183_Library$ export GITHUB_WORKSPACE=$(pwd)
-   ```
-5. Remove everything in test library, and re-create it
-   ```bash
-   adafruit@adafruit:~/Adafruit_BMP183_Library$ rm -rf ~/Arduino/libraries/Adafruit_Test_Library/; mkdir ~/Arduino/libraries/Adafruit_Test_Library
-   ```
-6. Still in the top-level directory of the library you'll be testing, copy the current library to Adafruit_Test_Library
-   ```bash
-   adafruit@adafruit:~/Adafruit_BMP183_Library$ cp * ~/Arduino/libraries/Adafruit_Test_Library/
-   ```
-7. Grep for build_platform.py in githubci.yml to find out what boards to test.
-   ```bash
-   adafruit@adafruit:~/Adafruit_BMP183_Library$ grep 'build_platform.py' .github/workflows/githubci.yml
-           run: python3 ci/build_platform.py main_platforms
-   ```
-8. Run test-platforms. This may take a while, and tests for some boards sometimes run orders of magnitude slower than tests for other boards.
-   ```bash
-   test-platforms main_platforms
-   ```
-   OR, if githubci.yml specified other boards, let's say the metro m0 and pyportal, you'd run this:
-   ```bash
-   test-platforms metro_m0 pyportal
-   ```
-
-Here's what that returns
-```bash
-adafruit@adafruit:~/Adafruit_BMP183_Library$ test-platforms main_platforms
-build dir: /home/dherrada/Adafruit_BMP183_Library
-
-########################################
-INSTALLING ARDUINO BOARDS
-########################################
-arduino-cli core update-index --additional-urls https://adafruit.github.io/arduino-board-index/package_adafruit_index.json,http://arduino.esp8266.com/stable/package_esp8266com_index.json,https://dl.espressif.com/dl/package_esp32_index.json,https://sandeepmistry.github.io/arduino-nRF5/package_nRF5_boards_index.json > /dev/null
-
-Installing Adafruit Unified Sensor
-arduino-cli lib install "Adafruit Unified Sensor" > /dev/null
-arduino-cli lib uninstall "Adafruit BMP183 Library"
-Library Adafruit BMP183 Library is not installed
-Libraries installed:  ['/home/adafruit/Arduino/libraries/Adafruit_VL53L0X', '/home/adafruit/Arduino/libraries/RTClib', '/home/adafruit/Arduino/libraries/Adafruit-RGB-LCD-Shield-Library', '/home/adafruit/Arduino/libraries/readme.txt', '/home/adafruit/Arduino/libraries/Adafruit_Test_Library', '/home/adafruit/Arduino/libraries/Adafruit_Unified_Sensor']
-  elif isinstance(platform, collections.Iterable):
-################################################################################
-SWITCHING TO arduino:avr:uno
-Installing arduino:avr ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO arduino:avr:leonardo
-Installing arduino:avr ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO arduino:avr:mega:cpu=atmega2560
-Installing arduino:avr ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO arduino:samd:arduino_zero_native
-Installing arduino:samd ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO esp8266:esp8266:huzzah:eesz=4M3M,xtal=80
-Installing esp8266:esp8266 ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO esp32:esp32:featheresp32:FlashFreq=80
-Installing esp32:esp32 ✓
-################################################################################
-  BMP183test.ino ✓
-################################################################################
-SWITCHING TO adafruit:samd:adafruit_metro_m4:speed=120
-Installing adafruit:samd ✓
-################################################################################
-	BMP183test.ino ✓
-################################################################################
-SWITCHING TO adafruit:nrf52:feather52840:softdevice=s140v6,debug=l0
-Installing adafruit:nrf52 ✓
-################################################################################
-	BMP183test.ino ✓
-```

actions_install.sh

@@ -3,10 +3,20 @@
 set -e
 
 pip3 install clint pyserial setuptools adafruit-nrfutil
-sudo apt-get update
+
-sudo apt-get install -y libllvm8 -V
+# Only install stuff if it is really missing. This should never be executed,
-sudo apt install -fy cppcheck clang-format-8
+# as the default image contains clang-format v10, v11 (default) and v12.
+# https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-Readme.md#language-and-runtime
 if [ ! -f /usr/bin/clang-format ]; then
+    sudo gem install apt-spy2
+    sudo apt-spy2 check
+    sudo apt-spy2 fix --commit
+
+    # after selecting a specific mirror, we need to run 'apt-get update'
+    sudo apt-get -o Acquire::Retries=3 update
+
+    sudo apt-get -o Acquire::Retries=3 install -y clang-format-8 libllvm8
+
     sudo ln -s /usr/bin/clang-format-8 /usr/bin/clang-format
 fi
 
@@ -18,6 +28,19 @@ mkdir ${HOME}/Arduino/libraries
 
 # install arduino IDE
 export PATH=$PATH:$GITHUB_WORKSPACE/bin
-curl -fsSL https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh | sh -s 0.18.3 2>&1
+echo $GITHUB_WORKSPACE/bin >> $GITHUB_PATH
+curl -fsSL https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh | sh
 arduino-cli config init > /dev/null
 arduino-cli core update-index > /dev/null
+
+# warn if this library does not have arduino-library tag in its topic
+case "$GITHUB_REPOSITORY" in
+(*/ci-arduino|*/Adafruit_Learning_System_Guides) ;;
+(*)
+  repo_topics=$(curl -f --request GET --url "https://api.github.com/repos/$GITHUB_REPOSITORY" || echo '{"topics":[]}')
+  repo_topics=$(echo $repo_topics | jq -r '.topics[]' )
+  if [[ ! $repo_topics =~ "arduino-library" ]]; then
+    echo "::warning::arduino-library is not found in this repo topics. Please add this tag in repo About"
+  fi
+esac
+

all_platforms.py

@@ -0,0 +1,179 @@
+# board: [ platform, uf2_family, manual core URL]
+ALL_PLATFORMS={
+    # classic Arduino AVR
+    "uno" : ["arduino:avr:uno", None, None],
+    "leonardo" : ["arduino:avr:leonardo", None, None],
+    "mega2560" : ["arduino:avr:mega:cpu=atmega2560", None, None],
+    # Arduino SAMD
+    "zero" : ["arduino:samd:arduino_zero_native", "0x68ed2b88", None, None],
+    "cpx" : ["arduino:samd:adafruit_circuitplayground_m0", "0x68ed2b88", None],
+    # Arduino MBED GIGA
+    "giga" : ["arduino:mbed_giga:giga", None, None],
+    # Espressif
+    "esp8266" : ["esp8266:esp8266:huzzah:eesz=4M3M,xtal=80", None, None],
+    "esp32" : ["esp32:esp32:featheresp32:FlashFreq=80", None, None],
+    "itsybitsy_esp32" : ["esp32:esp32:adafruit_itsybitsy_esp32:FlashFreq=80", None, None],
+    "feather_esp8266" : ["esp8266:esp8266:huzzah:xtal=80,vt=flash,exception=disabled,stacksmash=disabled,ssl=all,mmu=3232,non32xfer=fast,eesz=4M2M,ip=lm2f,dbg=Disabled,lvl=None____,wipe=none,baud=115200", None, None],
+    "feather_esp32" : ["esp32:esp32:featheresp32:FlashFreq=80", None, None],
+    "wippersnapper_feather_esp32" : ["esp32:esp32:featheresp32:FlashFreq=80,PartitionScheme=min_spiffs", None, None],
+    "feather_esp32_v2" : ["esp32:esp32:adafruit_feather_esp32_v2", None, None],
+    "qtpy_esp32" : ["esp32:esp32:adafruit_qtpy_esp32_pico", None, None],
+    "sparklemotion_esp32" : ["esp32:esp32:sparklemotion", None, None],
+    "sparklemotionmini_esp32" : ["esp32:esp32:sparklemotionmini", None, None],
+    ## ESP32-C3/C6
+    "feather_esp32c6" : ["esp32:esp32:adafruit_feather_esp32c6:FlashMode=qio", None, None],
+    "wippersnapper_feather_esp32c6" : ["esp32:esp32:adafruit_feather_esp32c6:CDCOnBoot=cdc,CPUFreq=160,FlashFreq=80,FlashMode=qio,PartitionScheme=min_spiffs", None, None],
+    "wippersnapper_feather_esp32c6_debug" : ["esp32:esp32:adafruit_feather_esp32c6:CDCOnBoot=cdc,DebugLevel=verbose,CPUFreq=160,FlashFreq=80,FlashMode=qio,PartitionScheme=min_spiffs", None, None],
+    "qtpy_esp32c3" : ["esp32:esp32:adafruit_qtpy_esp32c3:FlashMode=qio", None, None],
+    "wippersnapper_qtpy_esp32c3" : ["esp32:esp32:adafruit_qtpy_esp32c3:FlashMode=qio,PartitionScheme=min_spiffs", None, None],
+    ## ESP32-S2
+    "magtag" : ["esp32:esp32:adafruit_magtag29_esp32s2", "0xbfdd4eee", None],
+    "funhouse" : ["esp32:esp32:adafruit_funhouse_esp32s2", "0xbfdd4eee", None],
+    "funhouse_noota" : ["esp32:esp32:adafruit_funhouse_esp32s2:PartitionScheme=tinyuf2_noota", "0xbfdd4eee", None],
+    "metroesp32s2" : ["esp32:esp32:adafruit_metro_esp32s2", "0xbfdd4eee", None],
+    "qtpy_esp32s2" : ["esp32:esp32:adafruit_qtpy_esp32s2", "0xbfdd4eee", None],
+    "feather_esp32s2" : ["esp32:esp32:adafruit_feather_esp32s2", "0xbfdd4eee", None],
+    "feather_esp32s2_debug" : ["esp32:esp32:adafruit_feather_esp32s2:DebugLevel=verbose", "0xbfdd4eee", None],
+    "feather_esp32s2_tft" : ["esp32:esp32:adafruit_feather_esp32s2_tft", "0xbfdd4eee", None],
+    "feather_esp32s2_tft_debug" : ["esp32:esp32:adafruit_feather_esp32s2_tft:DebugLevel=verbose", "0xbfdd4eee", None],
+    "feather_esp32s2_reverse_tft" : ["esp32:esp32:adafruit_feather_esp32s2_reversetft", "0xbfdd4eee", None],
+    ## ESP32-S3
+    "feather_esp32s3" : ["esp32:esp32:adafruit_feather_esp32s3_nopsram", "0xc47e5767", None],
+    "feather_esp32s3_debug" : ["esp32:esp32:adafruit_feather_esp32s3_nopsram:DebugLevel=verbose", "0xc47e5767", None],
+    "feather_esp32s3_4mbflash_2mbpsram" : ["esp32:esp32:adafruit_feather_esp32s3", "0xc47e5767", None],
+    "feather_esp32s3_4mbflash_2mbpsram_debug" : ["esp32:esp32:adafruit_feather_esp32s3:DebugLevel=verbose", "0xc47e5767", None],
+    "feather_esp32s3_tft" : ["esp32:esp32:adafruit_feather_esp32s3_tft", "0xc47e5767", None],
+    "feather_esp32s3_tft_debug" : ["esp32:esp32:adafruit_feather_esp32s3_tft:DebugLevel=verbose", "0xc47e5767", None],
+    "feather_esp32s3_reverse_tft" : ["esp32:esp32:adafruit_feather_esp32s3_reversetft", "0xc47e5767", None],
+    "feather_esp32s3_reverse_tft_debug" : ["esp32:esp32:adafruit_feather_esp32s3_reversetft:DebugLevel=verbose", "0xc47e5767", None],
+    "matrixportal_s3" : ["esp32:esp32:adafruit_matrixportal_esp32s3", "0xc47e5767", None],
+    "metro_esp32s3" : ["esp32:esp32:adafruit_metro_esp32s3", "0xc47e5767", None],
+    "pycamera_s3" : ["esp32:esp32:adafruit_camera_esp32s3", "0xc47e5767", None],
+    "qualia_s3_rgb666" : ["esp32:esp32:adafruit_qualia_s3_rgb666", "0xc47e5767", None],
+    "qtpy_esp32s3" : ["esp32:esp32:adafruit_qtpy_esp32s3_nopsram", "0xc47e5767", None],
+    "qtpy_esp32s3_n4r2" : ["esp32:esp32:adafruit_qtpy_esp32s3_n4r2", "0xc47e5767", None],
+    # ESP32-P4
+    "esp32p4" : ["esp32:esp32:esp32p4:JTAGAdapter=default,PSRAM=disabled,USBMode=default,CDCOnBoot=cdc,MSCOnBoot=default,DFUOnBoot=default,UploadMode=default,PartitionScheme=default,CPUFreq=360,FlashMode=qio,FlashFreq=80,FlashSize=4M,UploadSpeed=921600,DebugLevel=none,EraseFlash=none", "0x3d308e94", None],
+    # Adafruit AVR
+    "trinket_3v" : ["adafruit:avr:trinket3", None, None],
+    "trinket_5v" : ["adafruit:avr:trinket5", None, None],
+    "protrinket_3v" : ["adafruit:avr:protrinket3", None, None],
+    "protrinket_5v" : ["adafruit:avr:protrinket5", None, None],
+    "gemma" : ["adafruit:avr:gemma", None, None],
+    "flora" : ["adafruit:avr:flora8", None, None],
+    "feather32u4" : ["adafruit:avr:feather32u4", None, None],
+    "cpc" : ["arduino:avr:circuitplay32u4cat", None, None],
+    # Adafruit SAMD
+    "gemma_m0" : ["adafruit:samd:adafruit_gemma_m0", "0x68ed2b88", None],
+    "trinket_m0" : ["adafruit:samd:adafruit_trinket_m0", "0x68ed2b88", None],
+    "feather_m0_express" : ["adafruit:samd:adafruit_feather_m0_express", "0x68ed2b88", None],
+    "feather_m0_express_tinyusb" : ["adafruit:samd:adafruit_feather_m0_express:usbstack=tinyusb", "0x68ed2b88", None],
+    "feather_m4_express" : ["adafruit:samd:adafruit_feather_m4:speed=120", "0x55114460", None],
+    "feather_m4_express_tinyusb" : ["adafruit:samd:adafruit_feather_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "feather_m4_can" : ["adafruit:samd:adafruit_feather_m4_can:speed=120", "0x55114460", None],
+    "feather_m4_can_tinyusb" : ["adafruit:samd:adafruit_feather_m4_can:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "metro_m0" : ["adafruit:samd:adafruit_metro_m0", "0x68ed2b88", None],
+    "metro_m0_tinyusb" : ["adafruit:samd:adafruit_metro_m0:usbstack=tinyusb", "0x68ed2b88", None],
+    "metro_m4" : ["adafruit:samd:adafruit_metro_m4:speed=120", "0x55114460", None],
+    "metro_m4_tinyusb" : ["adafruit:samd:adafruit_metro_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "metro_m4_airliftlite" : ["adafruit:samd:adafruit_metro_m4_airliftlite:speed=120", "0x55114460", None],
+    "metro_m4_airliftlite_tinyusb" : ["adafruit:samd:adafruit_metro_m4_airliftlite:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "pybadge" : ["adafruit:samd:adafruit_pybadge_m4:speed=120", "0x55114460", None],
+    "pybadge_tinyusb" : ["adafruit:samd:adafruit_pybadge_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "pygamer" : ["adafruit:samd:adafruit_pygamer_m4:speed=120", "0x55114460", None],
+    "pygamer_tinyusb" : ["adafruit:samd:adafruit_pygamer_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "hallowing_m0" : ["adafruit:samd:adafruit_hallowing", "0x68ed2b88", None],
+    "hallowing_m4" : ["adafruit:samd:adafruit_hallowing_m4:speed=120", "0x55114460", None],
+    "hallowing_m4_tinyusb" : ["adafruit:samd:adafruit_hallowing_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "neotrellis_m4" : ["adafruit:samd:adafruit_trellis_m4:speed=120", "0x55114460", None],
+    "monster_m4sk" : ["adafruit:samd:adafruit_monster_m4sk:speed=120", "0x55114460", None],
+    "monster_m4sk_tinyusb" : ["adafruit:samd:adafruit_monster_m4sk:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "pyportal" : ["adafruit:samd:adafruit_pyportal_m4:speed=120", "0x55114460", None],
+    "pyportal_tinyusb" : ["adafruit:samd:adafruit_pyportal_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "pyportal_titano" : ["adafruit:samd:adafruit_pyportal_m4_titano:speed=120", "0x55114460", None],
+    "pyportal_titano_tinyusb" : ["adafruit:samd:adafruit_pyportal_m4_titano:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "cpx_ada" : ["adafruit:samd:adafruit_circuitplayground_m0", "0x68ed2b88", None],
+    "grand_central" : ["adafruit:samd:adafruit_grandcentral_m4:speed=120", "0x55114460", None],
+    "grand_central_tinyusb" : ["adafruit:samd:adafruit_grandcentral_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "matrixportal" : ["adafruit:samd:adafruit_matrixportal_m4:speed=120", "0x55114460", None],
+    "matrixportal_tinyusb" : ["adafruit:samd:adafruit_matrixportal_m4:speed=120,usbstack=tinyusb", "0x55114460", None],
+    "neotrinkey_m0" : ["adafruit:samd:adafruit_neotrinkey_m0", "0x68ed2b88", None],
+    "rotarytrinkey_m0" : ["adafruit:samd:adafruit_rotarytrinkey_m0", "0x68ed2b88", None],
+    "neokeytrinkey_m0" : ["adafruit:samd:adafruit_neokeytrinkey_m0", "0x68ed2b88", None],
+    "slidetrinkey_m0" : ["adafruit:samd:adafruit_slidetrinkey_m0", "0x68ed2b88", None],
+    "proxlighttrinkey_m0" : ["adafruit:samd:adafruit_proxlighttrinkey_m0", "0x68ed2b88", None],
+    "trrstrinkey_m0" : ["adafruit:samd:adafruit_trrstrinkey_m0", "0x68ed2b88", None],
+    "thumbsticktrinkey_m0" : ["adafruit:samd:adafruit_thumbsticktrinkey_m0", "0x68ed2b88", None],
+    "sht4xtrinkey_m0" : ["adafruit:samd:adafruit_sht4xtrinkey_m0", "0x68ed2b88", None],
+    "pixeltrinkey_m0" : ["adafruit:samd:adafruit_pixeltrinkey_m0", "0x68ed2b88", None],
+    "X_m0" : ["adafruit:samd:adafruit_X_m0", "0x68ed2b88", None],
+    "qtpy_m0" : ["adafruit:samd:adafruit_qtpy_m0", "0x68ed2b88", None],
+    "qtpy_m0_tinyusb" : ["adafruit:samd:adafruit_qtpy_m0:usbstack=tinyusb", "0x68ed2b88", None],
+    # Arduino SAMD
+    "mkrwifi1010" : ["arduino:samd:mkrwifi1010", "0x8054", None],
+    "nano_33_iot" : ["arduino:samd:nano_33_iot", "0x8057", None],
+    # Arduino nRF
+    "microbit" : ["sandeepmistry:nRF5:BBCmicrobit:softdevice=s110", None, None],
+    # Adafruit nRF
+    "nrf52832" : ["adafruit:nrf52:feather52832:softdevice=s132v6,debug=l0", None, None],
+    "nrf52840" : ["adafruit:nrf52:feather52840:softdevice=s140v6,debug=l0", "0xada52840", None],
+    "cpb" : ["adafruit:nrf52:cplaynrf52840:softdevice=s140v6,debug=l0", "0xada52840", None],
+    "clue" : ["adafruit:nrf52:cluenrf52840:softdevice=s140v6,debug=l0", "0xada52840", None],
+    "ledglasses_nrf52840" : ["adafruit:nrf52:ledglasses_nrf52840:softdevice=s140v6,debug=l0", "0xada52840", None],
+    # RP2040 & RP2350 (Philhower)
+    "pico_rp2040" : ["rp2040:rp2040:rpipico:freq=125,flash=2097152_0", "0xe48bff56", None],
+    "pico_rp2040_tinyusb" : ["rp2040:rp2040:rpipico:flash=2097152_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "pico_rp2040_tinyusb_host" : ["rp2040:rp2040:rpipico:flash=2097152_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb_host", "0xe48bff56", None],
+    "pico_rp2350" : ["rp2040:rp2040:rpipico2:freq=125,flash=4194304_0", "0xe48bff56", None],
+    "pico_rp2350_tinyusb" : ["rp2040:rp2040:rpipico2:flash=4194304_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "pico_rp2350_tinyusb_host" : ["rp2040:rp2040:rpipico2:flash=4194304_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb_host", "0xe48bff56", None],
+    "picow_rp2040" : ["rp2040:rp2040:rpipicow:flash=2097152_0,freq=125", "0xe48bff56", None],
+    "picow_rp2040_tinyusb" : ["rp2040:rp2040:rpipicow:flash=2097152_131072,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040" : ["rp2040:rp2040:adafruit_feather:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_feather:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_adalogger" : ["rp2040:rp2040:adafruit_feather_adalogger:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_adalogger_tinyusb" : ["rp2040:rp2040:adafruit_feather_adalogger:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_rfm" : ["rp2040:rp2040:adafruit_feather_rfm:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_rfm_tinyusb" : ["rp2040:rp2040:adafruit_feather_rfm:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_dvi" : ["rp2040:rp2040:adafruit_feather_dvi:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "feather_rp2040_dvi_tinyusb" : ["rp2040:rp2040:adafruit_feather_dvi:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2040_usbhost_tinyusb" : ["rp2040:rp2040:adafruit_feather_usb_host:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "feather_rp2350" : ["rp2040:rp2040:adafruit_feather_rp2350_hstx:arch=arm,flash=8388608_0,freq=150,dbgport=Disabled,dbglvl=None,usbstack=picosdk", "0xe48bff56", None],
+    "feather_rp2350_tinyusb" : ["rp2040:rp2040:adafruit_feather_rp2350_hstx:arch=arm,flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "metro_rp2040" : ["rp2040:rp2040:adafruit_metro:flash=16777216_0,freq=200,dbgport=Disabled,dbglvl=None,usbstack=picosdk", "0xe48bff56", None],
+    "metro_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_metro:flash=16777216_0,freq=200,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "metro_rp2350" : ["rp2040:rp2040:adafruit_metro_rp2350:arch=arm,flash=16777216_0,freq=150,dbgport=Disabled,dbglvl=None,usbstack=picosdk", "0xe48bff56", None],
+    "metro_rp2350_tinyusb" : ["rp2040:rp2040:adafruit_metro_rp2350:arch=arm,flash=16777216_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "qt2040_trinkey" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:flash=8388608_0,freq=125","0xe48bff56", None],
+    "qt2040_trinkey_tinyusb" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "qt_py_rp2040": ["rp2040:rp2040:adafruit_qtpy:freq=125,flash=8388608_0", "0xe48bff56", None],
+    "qt_py_rp2040_tinyusb": ["rp2040:rp2040:adafruit_qtpy:flash=8388608_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "itsybitsy_rp2040" : ["rp2040:rp2040:adafruit_itsybitsy:freq=125,flash=8388608_524288", "0xe48bff56", None],
+    "itsybitsy_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_itsybitsy:flash=8388608_524288,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "floppsy_rp2040" : ["rp2040:rp2040:adafruit_floppsy:freq=125,flash=16777216_14680064", "0xe48bff56", None],
+    "floppsy_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_floppsy:flash=16777216_14680064,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    "fruit_jam" : ["rp2040:rp2040:adafruit_fruitjam:arch=arm,flash=16777216_0,freq=150,dbgport=Disabled,dbglvl=None,usbstack=picosdk", "0xe48bff56", None],
+    "fruit_jam_tinyusb" : ["rp2040:rp2040:adafruit_fruitjam:arch=arm,flash=16777216_0,freq=120,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56", None],
+    # Attiny8xy, 16xy, 32xy (SpenceKonde)
+    "attiny3217" : ["megaTinyCore:megaavr:atxy7:chip=3217", None, None],
+    "attiny3216" : ["megaTinyCore:megaavr:atxy6:chip=3216", None, None],
+    "attiny1617" : ["megaTinyCore:megaavr:atxy7:chip=1617", None, None],
+    "attiny1616" : ["megaTinyCore:megaavr:atxy6:chip=1616", None, None],
+    "attiny1607" : ["megaTinyCore:megaavr:atxy7:chip=1607", None, None],
+    "attiny1606" : ["megaTinyCore:megaavr:atxy6:chip=1606", None, None],
+    "attiny817" : ["megaTinyCore:megaavr:atxy7:chip=817", None, None],
+    "attiny816" : ["megaTinyCore:megaavr:atxy6:chip=816", None, None],
+    "attiny807" : ["megaTinyCore:megaavr:atxy7:chip=807", None, None],
+    "attiny806" : ["megaTinyCore:megaavr:atxy6:chip=806", None, None],
+
+    # CH32v2 (openwch)
+    "CH32V20x_EVT": ["WCH:ch32v:CH32V20x_EVT", None, None],
+
+    # groupings
+    "main_platforms" : ("uno", "leonardo", "mega2560", "zero", "qtpy_m0",
+                        "esp8266", "esp32", "metro_m4", "trinket_m0"),
+    "arcada_platforms" : ("pybadge", "pygamer", "hallowing_m4",
+                          "cpb", "cpx_ada"),
+    "wippersnapper_platforms" : ("metro_m4_airliftlite_tinyusb", "pyportal_tinyusb"),
+    "rp2040_platforms" : ("pico_rp2040", "feather_rp2040", "feather_rp2350")
+}

build_platform.py

@@ -2,8 +2,12 @@ import sys
 import glob
 import time
 import os
+import re
+import shutil
 import subprocess
 import collections
+from contextlib import contextmanager
+from all_platforms import ALL_PLATFORMS
 
 # optional wall option cause build failed if has warnings
 BUILD_WALL = False
@@ -16,6 +20,15 @@ if "--no_warn" in sys.argv:
     BUILD_WARN = False
     sys.argv.remove("--no_warn")
 
+# optional timeout argument to extend build time
+# for larger sketches or firmware builds
+BUILD_TIMEOUT = False
+if "--build_timeout" in sys.argv:
+    BUILD_TIMEOUT = True
+    popen_timeout = int(sys.argv[sys.argv.index("--build_timeout") + 1])
+    sys.argv.pop(sys.argv.index("--build_timeout") + 1)
+    sys.argv.remove("--build_timeout")
+
 # add user bin to path!
 BUILD_DIR = ''
 # add user bin to path!
@@ -38,6 +51,7 @@ IS_LEARNING_SYS = False
 if "Adafruit_Learning_System_Guides" in BUILD_DIR:
     print("Found learning system repo")
     IS_LEARNING_SYS = True
+    shutil.rmtree(BUILD_DIR + "/ci/examples/Blink")
 elif "METROX-Examples-and-Project-Sketches" in BUILD_DIR:
     print("Found MetroX Examples Repo")
     IS_LEARNING_SYS = True
@@ -48,94 +62,20 @@ elif "METROX-Examples-and-Project-Sketches" in BUILD_DIR:
 CROSS = u'\N{cross mark}'
 CHECK = u'\N{check mark}'
 
-ALL_PLATFORMS={
+BSP_URLS = (
-    # classic Arduino AVR
+    "https://adafruit.github.io/arduino-board-index/package_adafruit_index.json,"
-    "uno" : ["arduino:avr:uno", None],
+    "http://arduino.esp8266.com/stable/package_esp8266com_index.json,"
-    "leonardo" : ["arduino:avr:leonardo", None],
+    #"https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_dev_index.json," # esp32 beta release
-    "mega2560" : ["arduino:avr:mega:cpu=atmega2560", None],
+    "https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json,"
-    # Arduino SAMD
+    "https://sandeepmistry.github.io/arduino-nRF5/package_nRF5_boards_index.json,"
-    "zero" : ["arduino:samd:arduino_zero_native", "0x68ed2b88"],
+    "https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json,"
-    "cpx" : ["arduino:samd:adafruit_circuitplayground_m0", "0x68ed2b88"],
+    "https://drazzy.good-enough.cloud/package_drazzy.com_index.json,"
-    # Espressif
+    "https://github.com/openwch/board_manager_files/raw/main/package_ch32v_index.json"
-    "esp8266" : ["esp8266:esp8266:huzzah:eesz=4M3M,xtal=80", None],
+)
-    "esp32" : ["esp32:esp32:featheresp32:FlashFreq=80", None],
+
-    "magtag" : ["esp32:esp32:adafruit_magtag29_esp32s2", "0xbfdd4eee"],
+# global exit code
-    "funhouse" : ["esp32:esp32:adafruit_funhouse_esp32s2", "0xbfdd4eee"],
+success = 0
-    "metroesp32s2" : ["esp32:esp32:adafruit_metro_esp32s2", "0xbfdd4eee"],
-    # Adafruit AVR
-    "trinket_3v" : ["adafruit:avr:trinket3", None],
-    "trinket_5v" : ["adafruit:avr:trinket5", None],
-    "protrinket_3v" : ["adafruit:avr:protrinket3", None],
-    "protrinket_5v" : ["adafruit:avr:protrinket5", None],
-    "gemma" : ["adafruit:avr:gemma", None],
-    "flora" : ["adafruit:avr:flora8", None],
-    "feather32u4" : ["adafruit:avr:feather32u4", None],
-    "cpc" : ["arduino:avr:circuitplay32u4cat", None],
-    # Adafruit SAMD
-    "gemma_m0" : ["adafruit:samd:adafruit_gemma_m0", "0x68ed2b88"],
-    "trinket_m0" : ["adafruit:samd:adafruit_trinket_m0", "0x68ed2b88"],
-    "feather_m0_express" : ["adafruit:samd:adafruit_feather_m0_express", "0x68ed2b88"],
-    "feather_m4_can" : ["adafruit:samd:adafruit_feather_m4_can:speed=120", "0x68ed2b88"],
-    "feather_m4_can_tinyusb" : ["adafruit:samd:adafruit_feather_m4_can:speed=120,usbstack=tinyusb", "0x68ed2b88"],
-    "metro_m0" : ["adafruit:samd:adafruit_metro_m0", "0x68ed2b88"],
-    "metro_m0_tinyusb" : ["adafruit:samd:adafruit_metro_m0:usbstack=tinyusb", "0x68ed2b88"],
-    "metro_m4" : ["adafruit:samd:adafruit_metro_m4:speed=120", "0x55114460"],
-    "metro_m4_tinyusb" : ["adafruit:samd:adafruit_metro_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "metro_m4_airliftlite" : ["adafruit:samd:adafruit_metro_m4_airliftlite:speed=120", "0x55114460"],
-    "metro_m4_airliftlite_tinyusb" : ["adafruit:samd:adafruit_metro_m4_airliftlite:speed=120,usbstack=tinyusb", "0x55114460"],
-    "pybadge" : ["adafruit:samd:adafruit_pybadge_m4:speed=120", "0x55114460"],
-    "pybadge_tinyusb" : ["adafruit:samd:adafruit_pybadge_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "pygamer" : ["adafruit:samd:adafruit_pygamer_m4:speed=120", "0x55114460"],
-    "hallowing_m0" : ["adafruit:samd:adafruit_hallowing", "0x68ed2b88"],
-    "hallowing_m4" : ["adafruit:samd:adafruit_hallowing_m4:speed=120", "0x55114460"],
-    "hallowing_m4_tinyusb" : ["adafruit:samd:adafruit_hallowing_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "neotrellis_m4" : ["adafruit:samd:adafruit_trellis_m4:speed=120", "0x55114460"],
-    "monster_m4sk" : ["adafruit:samd:adafruit_monster_m4sk:speed=120", "0x55114460"],
-    "monster_m4sk_tinyusb" : ["adafruit:samd:adafruit_monster_m4sk:speed=120,usbstack=tinyusb", "0x55114460"],
-    "pyportal" : ["adafruit:samd:adafruit_pyportal_m4:speed=120", "0x55114460"],
-    "pyportal_tinyusb" : ["adafruit:samd:adafruit_pyportal_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "pyportal_titano" : ["adafruit:samd:adafruit_pyportal_m4_titano:speed=120", "0x55114460"],
-    "pyportal_titano_tinyusb" : ["adafruit:samd:adafruit_pyportal_m4_titano:speed=120,usbstack=tinyusb", "0x55114460"],
-    "cpx_ada" : ["adafruit:samd:adafruit_circuitplayground_m0", "0x68ed2b88"],
-    "grand_central" : ["adafruit:samd:adafruit_grandcentral_m4:speed=120", "0x55114460"],
-    "grand_central_tinyusb" : ["adafruit:samd:adafruit_grandcentral_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "matrixportal" : ["adafruit:samd:adafruit_matrixportal_m4:speed=120", "0x55114460"],
-    "matrixportal_tinyusb" : ["adafruit:samd:adafruit_matrixportal_m4:speed=120,usbstack=tinyusb", "0x55114460"],
-    "neotrinkey_m0" : ["adafruit:samd:adafruit_neotrinkey_m0", "0x68ed2b88"],
-    "rotarytrinkey_m0" : ["adafruit:samd:adafruit_rotarytrinkey_m0", "0x68ed2b88"],
-    "neokeytrinkey_m0" : ["adafruit:samd:adafruit_neokeytrinkey_m0", "0x68ed2b88"],
-    "slidetrinkey_m0" : ["adafruit:samd:adafruit_slidetrinkey_m0", "0x68ed2b88"],
-    "proxlighttrinkey_m0" : ["adafruit:samd:adafruit_proxlighttrinkey_m0", "0x68ed2b88"],
-    "qtpy_m0" : ["adafruit:samd:adafruit_qtpy_m0", "0x68ed2b88"],
-    # Arduino nRF
-    "microbit" : ["sandeepmistry:nRF5:BBCmicrobit:softdevice=s110", None],
-    # Adafruit nRF
-    "nrf52832" : ["adafruit:nrf52:feather52832:softdevice=s132v6,debug=l0", None],
-    "nrf52840" : ["adafruit:nrf52:feather52840:softdevice=s140v6,debug=l0", "0xada52840"],
-    "cpb" : ["adafruit:nrf52:cplaynrf52840:softdevice=s140v6,debug=l0", "0xada52840"],
-    "clue" : ["adafruit:nrf52:cluenrf52840:softdevice=s140v6,debug=l0", "0xada52840"],
-    # RP2040 (Philhower)
-    "pico_rp2040" : ["rp2040:rp2040:rpipico:freq=125,flash=2097152_0", "0xe48bff56"],
-    "pico_rp2040_tinyusb" : ["rp2040:rp2040:rpipico:flash=2097152_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56"],
-    "feather_rp2040" : ["rp2040:rp2040:adafruitfeather:freq=125,flash=8388608_0", "0xe48bff56"],
-    "feather_rp2040_tinyusb" : ["rp2040:rp2040:adafruit_feather:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56"],
-    "qt2040_trinkey" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:freq=125,flash=8388608_0", "0xe48bff56"],
-    "qt2040_trinkey_tinyusb" : ["rp2040:rp2040:adafruit_trinkeyrp2040qt:flash=8388608_0,freq=125,dbgport=Disabled,dbglvl=None,usbstack=tinyusb", "0xe48bff56"],
-    # Attiny8xy (SpenceKonde)
-    "attiny817" : ["megaTinyCore:megaavr:atxy7:chip=817", None],
-    "attiny816" : ["megaTinyCore:megaavr:atxy6:chip=816", None],
-    "attiny807" : ["megaTinyCore:megaavr:atxy7:chip=807", None],
-    "attiny806" : ["megaTinyCore:megaavr:atxy6:chip=806", None],
-    # groupings
-    "main_platforms" : ("uno", "leonardo", "mega2560", "zero", "qtpy_m0",
-                        "esp8266", "esp32", "metro_m4", "trinket_m0"),
-    "arcada_platforms" : ("pybadge", "pygamer", "hallowing_m4",
-                          "cpb", "cpx_ada"),
-    "wippersnapper_platforms" : ("metro_m4_airliftlite_tinyusb", "pyportal_tinyusb"),
-    "rp2040_platforms" : ("pico_rp2040", "feather_rp2040")
-}
 
-BSP_URLS = "https://adafruit.github.io/arduino-board-index/package_adafruit_index.json,http://arduino.esp8266.com/stable/package_esp8266com_index.json,https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_dev_index.json,https://sandeepmistry.github.io/arduino-nRF5/package_nRF5_boards_index.json,https://github.com/earlephilhower/arduino-pico/releases/download/global/package_rp2040_index.json,http://drazzy.com/package_drazzy.com_index.json"
 
 class ColorPrint:
 
@@ -160,32 +100,98 @@ class ColorPrint:
         sys.stdout.write('\x1b[1;37m' + message.strip() + '\x1b[0m' + end)
 
 
-def install_platform(platform):
+def manually_install_esp32_bsp(repo_info):
-    print("Installing", platform, end=" ")
+    print("Manually installing latest ESP32 BSP...")
-    if platform == "adafruit:avr":   # we have a platform dep
+    # Assemble git url
-        install_platform("arduino:avr")
+    repo_url = "git clone -b {0} https://github.com/{1}/arduino-esp32.git esp32".format(repo_info.split("/")[1], repo_info.split("/")[0])
-    if os.system("arduino-cli core install "+platform+" --additional-urls "+BSP_URLS+" > /dev/null") != 0:
+    # Locally clone repo (https://docs.espressif.com/projects/arduino-esp32/en/latest/installing.html#linux)
-        ColorPrint.print_fail("FAILED to install "+platform)
+    os.system("mkdir -p /home/runner/Arduino/hardware/espressif")
+    print("Cloning %s"%repo_url)
+    cmd = "cd /home/runner/Arduino/hardware/espressif && " + repo_url
+    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+    r = proc.wait(timeout=1000)
+    out = proc.stdout.read()
+    err = proc.stderr.read()
+    if r != 0:
+        ColorPrint.print_fail("Failed to download ESP32 Arduino BSP!")
+        ColorPrint.print_fail(out.decode("utf-8"))
+        ColorPrint.print_fail(err.decode("utf-8"))
+        exit(-1)
+    print(out)
+    print("Cloned repository!")
+
+    print("Installing ESP32 Arduino BSP...")
+    cmd = "cd /home/runner/Arduino/hardware/espressif/esp32/tools && python3 get.py"
+    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
+    r = proc.wait(timeout=1000)
+    out = proc.stdout.read()
+    err = proc.stderr.read()
+    if r != 0:
+        ColorPrint.print_fail("Failed to install ESP32 Arduino BSP!")
+        ColorPrint.print_fail(out.decode("utf-8"))
+        ColorPrint.print_fail(err.decode("utf-8"))
+        exit(-1)
+    print(out)
+    print("Installed ESP32 BSP from source!")
+
+
+def install_platform(fqbn, full_platform_name=None):
+    if os.path.exists("/home/runner/.arduino15/package_drazzy.json"):
+        print("Moving drazzy.json")
+        shutil.move("/home/runner/.arduino15/package_drazzy.json", "/home/runner/.arduino15/package_drazzy.com_index.json")
+    print("Installing", fqbn, end=" ")
+    if fqbn == "adafruit:avr":   # we have a platform dep
+        install_platform("arduino:avr", full_platform_name)
+    if full_platform_name[2] is not None:
+        manually_install_esp32_bsp(full_platform_name[2]) # build esp32 bsp from desired source and branch
+    for retry in range(0, 3):
+        print("arduino-cli core install "+fqbn+" --additional-urls "+BSP_URLS)
+        if os.system("arduino-cli core install "+fqbn+" --additional-urls "+BSP_URLS+" > /dev/null") == 0:
+            break
+        print("...retrying...", end=" ")
+        time.sleep(10) # wait 10 seconds then try again?
+    else:
+        # tried 3 times to no avail
+        ColorPrint.print_fail("FAILED to install "+fqbn)
         exit(-1)
     ColorPrint.print_pass(CHECK)
+    # print installed core version
+    print(os.popen('arduino-cli core list | grep {}'.format(fqbn)).read(), end='')
+
 
 def run_or_die(cmd, error):
     print(cmd)
-    if os.system(cmd) != 0:
+    attempt = 0
+    while attempt < 3:
+        if os.system(cmd) == 0:
+            return
+        attempt += 1
+        print('attempt {} failed, {} retry left'.format(attempt, 3-attempt))
+        time.sleep(5)
     ColorPrint.print_fail(error)
     exit(-1)
 
-################################ Install Arduino IDE
+
+def is_library_installed(lib_name):
+    try:
+        installed_libs = subprocess.check_output(["arduino-cli", "lib", "list"]).decode("utf-8")
+        return not all(not item for item in [re.match('^'+lib_name+'\\s*\\d+\\.', line) for line in installed_libs.split('\n')])
+    except subprocess.CalledProcessError as e:
+        print("Error checking installed libraries:", e)
+        return False
+
+
+def install_library_deps():
     print()
     ColorPrint.print_info('#'*40)
-print("INSTALLING ARDUINO BOARDS")
+    print("INSTALLING ARDUINO LIBRARIES")
     ColorPrint.print_info('#'*40)
 
     run_or_die("arduino-cli core update-index --additional-urls "+BSP_URLS+
-           " > /dev/null", "FAILED to update core indecies")
+               " > /dev/null", "FAILED to update core indices")
     print()
 
-################################ Install dependancies
+    # Install dependencies
     our_name = None
     try:
         if IS_LEARNING_SYS:
@@ -199,15 +205,18 @@ try:
                 deps = line.replace("depends=", "").split(",")
                 for dep in deps:
                     dep = dep.strip()
+                    if not is_library_installed(dep):
                         print("Installing "+dep)
                         run_or_die('arduino-cli lib install "'+dep+'" > /dev/null',
-                           "FAILED to install dependancy "+dep)
+                                   "FAILED to install dependency "+dep)
+                    else:
+                        print("Skipping already installed lib: "+dep)
     except OSError:
         print("No library dep or properties found!")
         pass  # no library properties
 
     # Delete the existing library if we somehow downloaded
-# due to dependancies
+    # due to dependencies
     if our_name:
         run_or_die("arduino-cli lib uninstall \""+our_name+"\"", "Could not uninstall")
 
@@ -222,12 +231,21 @@ if not IS_LEARNING_SYS:
 
 ################################ UF2 Utils.
 
+
+def glob01(pattern):
+    result = glob.glob(pattern)
+    if len(result) > 1:
+        raise RuntimeError(f"Required pattern {pattern} to match at most 1 file, got {result}")
+    return result[0] if result else None
+
+
 def glob1(pattern):
     result = glob.glob(pattern)
     if len(result) != 1:
         raise RuntimeError(f"Required pattern {pattern} to match exactly 1 file, got {result}")
     return result[0]
 
+
 def download_uf2_utils():
     """Downloads uf2conv tools if we don't already have them
     """
@@ -243,34 +261,174 @@ def download_uf2_utils():
         return False
     return True
 
-def generate_uf2(example_path):
-    """Generates a .uf2 file from a .bin or .hex file.
-    :param str example_path: A path to the compiled .bin or .hex file.
 
+def generate_uf2(platform, fqbn, example_path):
+    """Generates a .uf2 file from a .bin or .hex file.
+    :param str platform: The platform name.
+    :param str fqbn: The fully qualified board name.
+    :param str example_path: A path to the compiled .bin or .hex file.
     """
     if not download_uf2_utils():
-        return 1 # success = 1
+        return None
-    cli_build_path = "build/*.*." + fqbn.split(':')[2] + "/*.hex"
+
+    cli_build_uf2_path = "build/*.*." + fqbn.split(':')[2] + "/*.uf2"
+    uf2_input_file = glob01(os.path.join(example_path, cli_build_uf2_path))
+
+    # Some platforms, like rp2040, directly generate a uf2 file, so no need to do it ourselves
+    if uf2_input_file is not None:
+        output_file = os.path.splitext(uf2_input_file)[0] + ".uf2"
+        ColorPrint.print_pass(CHECK)
+        ColorPrint.print_info("Used uf2 generated by arduino-cli")
+        return output_file
+
+    # Generate using a hex file for all platforms except for ESP32-S2, ESP32-S3 (exports as .bin files)
+    if not any (x in fqbn for x in ["esp32s2", "esp32s3"]):
+        cli_build_hex_path = "build/*.*." + fqbn.split(':')[2] + "/*.hex"
+        hex_input_file = glob1(os.path.join(example_path, cli_build_hex_path))
+        output_file = os.path.splitext(hex_input_file)[0] + ".uf2"
+        family_id = ALL_PLATFORMS[platform][1]
+        cmd = ['python3', 'uf2conv.py', hex_input_file, '-c', '-f', family_id, '-o', output_file]
+    else:
+        cli_build_path = "build/*.*." + fqbn.split(':')[2] + "/*.ino.bin"
         input_file = glob1(os.path.join(example_path, cli_build_path))
         output_file = os.path.splitext(input_file)[0] + ".uf2"
         family_id = ALL_PLATFORMS[platform][1]
-    cmd = ['python3', 'uf2conv.py', input_file, '-c', '-f', family_id, '-o', output_file]
+        cmd = ['python3', 'uf2conv.py', input_file, '-c', '-f', family_id, '-b', "0x0000", '-o', output_file]
+
     proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    if BUILD_TIMEOUT:
+        r = proc.wait(timeout=popen_timeout)
+    else:
         r = proc.wait(timeout=60)
     out = proc.stdout.read()
     err = proc.stderr.read()
-    if r == 0 and not err:
+    if r == 0: # and not err:  # we might get warnings that do not affect the result
         ColorPrint.print_pass(CHECK)
         ColorPrint.print_info(out.decode("utf-8"))
     else:
         ColorPrint.print_fail(CROSS)
+        ColorPrint.print_fail("\n\rERRCODE:", str(r))
+        ColorPrint.print_fail("\n\rOUTPUT: ", out.decode("utf-8"))
+        ColorPrint.print_fail("\n\rERROR: ", err.decode("utf-8"))
+        return None
+    return output_file
+
+
+@contextmanager
+def group_output(title):
+    sys.stdout.flush()
+    sys.stderr.flush()
+    print(f"::group::{title}")
+    try:
+        yield
+    finally:
+        sys.stdout.flush()
+        sys.stderr.flush()
+        print(f"::endgroup::")
+        sys.stdout.flush()
+
+
+def test_examples_in_folder(platform, folderpath):
+    global success
+    fqbn = ALL_PLATFORMS[platform][0]
+    for example in sorted(os.listdir(folderpath)):
+        examplepath = folderpath+"/"+example
+        if os.path.isdir(examplepath):
+            test_examples_in_folder(platform, examplepath)
+            continue
+        if not examplepath.endswith(".ino"):
+            continue
+
+        print('\t'+example, end=' ')
+
+        # check if we should SKIP
+        skipfilename = folderpath+"/."+platform+".test.skip"
+        onlyfilename = folderpath+"/."+platform+".test.only"
+        # check if we should GENERATE UF2
+        gen_file_name = folderpath+"/."+platform+".generate"
+
+        # .skip txt include all skipped platforms, one per line
+        skip_txt = folderpath+"/.skip.txt"
+
+        is_skip = False
+        if os.path.exists(skipfilename):
+            is_skip = True
+        if os.path.exists(skip_txt):
+            with open(skip_txt) as f:
+                lines = f.readlines()
+                for line in lines:
+                    if line.strip() == platform:
+                        is_skip = True
+                        break
+        if is_skip:
+            ColorPrint.print_warn("skipping")
+            continue
+
+        if glob.glob(folderpath+"/.*.test.only"):
+            platformname = glob.glob(folderpath+"/.*.test.only")[0].split('.')[1]
+            if platformname != "none" and not platformname in ALL_PLATFORMS:
+                # uh oh, this isnt a valid testonly!
+                ColorPrint.print_fail(CROSS)
+                ColorPrint.print_fail("This example does not have a valid .platform.test.only file")
+                success = 1
+                continue
+            if not os.path.exists(onlyfilename):
+                ColorPrint.print_warn("skipping")
+                continue
+        if os.path.exists(gen_file_name):
+            ColorPrint.print_info("generating")
+
+        if BUILD_WARN:
+            if os.path.exists(gen_file_name):
+                cmd = ['arduino-cli', 'compile', '--warnings', 'all', '--fqbn', fqbn, '-e', folderpath]
+            else:
+                cmd = ['arduino-cli', 'compile', '--warnings', 'all', '--fqbn', fqbn, folderpath]
+        else:
+            cmd = ['arduino-cli', 'compile', '--warnings', 'none', '--export-binaries', '--fqbn', fqbn, folderpath]
+        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
+                                stderr=subprocess.PIPE)
+        try:
+            if BUILD_TIMEOUT:
+                out, err = proc.communicate(timeout=popen_timeout)
+            else:
+                out, err = proc.communicate(timeout=120)
+            r = proc.returncode
+        except:
+            proc.kill()
+            out, err = proc.communicate()
+            r = 1
+
+        if r == 0 and not (err and BUILD_WALL == True):
+            ColorPrint.print_pass(CHECK)
+            if err:
+                # also print out warning message
+                with group_output(f"{example} {fqbn} build output"):
+                    ColorPrint.print_fail(err.decode("utf-8"))
+            if os.path.exists(gen_file_name):
+                if ALL_PLATFORMS[platform][1] is None:
+                    ColorPrint.print_info("Platform does not support UF2 files, skipping...")
+                else:
+                    ColorPrint.print_info("Generating UF2...")
+                    filename = generate_uf2(platform, fqbn, folderpath)
+                    if filename is None:
+                        success = 1  # failure
+                    if IS_LEARNING_SYS:
+                        fqbnpath, uf2file = filename.split("/")[-2:]
+                        os.makedirs(BUILD_DIR+"/build", exist_ok=True)
+                        os.makedirs(BUILD_DIR+"/build/"+fqbnpath, exist_ok=True)
+                        shutil.copy(filename, BUILD_DIR+"/build/"+fqbnpath+"-"+uf2file)
+                        os.system("ls -lR "+BUILD_DIR+"/build")
+        else:
+            ColorPrint.print_fail(CROSS)
+            with group_output(f"{example} {fqbn} built output"):
                 ColorPrint.print_fail(out.decode("utf-8"))
                 ColorPrint.print_fail(err.decode("utf-8"))
-    return 0
+            success = 1
 
-################################ Test platforms
+
+def main():
+    # Test platforms
     platforms = []
-success = 0
 
     # expand groups:
     for arg in sys.argv[1:]:
@@ -284,107 +442,21 @@ for arg in sys.argv[1:]:
             print("Unknown platform: ", arg)
             exit(-1)
 
-def test_examples_in_folder(folderpath):
+    # Install libraries deps
-    global success
+    install_library_deps()
-    for example in sorted(os.listdir(folderpath)):
-        examplepath = folderpath+"/"+example
-        if os.path.isdir(examplepath):
-            test_examples_in_folder(examplepath)
-            continue
-        if not examplepath.endswith(".ino"):
-            continue
-
-        print('\t'+example, end=' ')
-        # check if we should SKIP
-        skipfilename = folderpath+"/."+platform+".test.skip"
-        onlyfilename = folderpath+"/."+platform+".test.only"
-        # check if we should GENERATE UF2
-        gen_file_name = folderpath+"/."+platform+".generate"
-        if os.path.exists(skipfilename):
-            ColorPrint.print_warn("skipping")
-            continue
-        if glob.glob(folderpath+"/.*.test.only") and not os.path.exists(onlyfilename):
-            ColorPrint.print_warn("skipping")
-            continue
-        if os.path.exists(gen_file_name):
-            ColorPrint.print_info("generating")
-
-        if BUILD_WARN:
-            if os.path.exists(gen_file_name):
-                cmd = ['arduino-cli', 'compile', '--warnings', 'all', '--fqbn', fqbn, '-e', examplepath]
-            else:
-                cmd = ['arduino-cli', 'compile', '--warnings', 'all', '--fqbn', fqbn, examplepath]
-        else:
-            cmd = ['arduino-cli', 'compile', '--warnings', 'none', '--export-binaries', '--fqbn', fqbn, examplepath]
-        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
-                                stderr=subprocess.PIPE)
-        r = proc.wait(timeout=60)
-        out = proc.stdout.read()
-        err = proc.stderr.read()
-        if r == 0 and not (err and BUILD_WALL == True):
-            ColorPrint.print_pass(CHECK)
-            if err:
-                # also print out warning message
-                ColorPrint.print_fail(err.decode("utf-8"))
-            if os.path.exists(gen_file_name):
-                if ALL_PLATFORMS[platform][1] == None:
-                    ColorPrint.print_info("Platform does not support UF2 files, skipping...")
-                else:
-                    ColorPrint.print_info("Generating UF2...")
-                    success = generate_uf2(folderpath)
-        else:
-            ColorPrint.print_fail(CROSS)
-            ColorPrint.print_fail(out.decode("utf-8"))
-            ColorPrint.print_fail(err.decode("utf-8"))
-            success = 1
-
-def test_examples_in_learningrepo(folderpath):
-    global success
-    for project in os.listdir(folderpath):
-        projectpath = folderpath+"/"+project
-        if os.path.isdir(learningrepo):
-            test_examples_in_learningrepo(projectpath)
-            continue
-        if not projectpath.endswith(".ino"):
-            continue
-	    # found an INO!
-        print('\t'+projectpath, end=' ', flush=True)
-        # check if we should SKIP
-        skipfilename = folderpath+"/."+platform+".test.skip"
-        onlyfilename = folderpath+"/."+platform+".test.only"
-        if os.path.exists(skipfilename):
-            ColorPrint.print_warn("skipping")
-            continue
-        elif glob.glob(folderpath+"/.*.test.only") and not os.path.exists(onlyfilename):
-            ColorPrint.print_warn("skipping")
-            continue
-
-        cmd = ['arduino-cli', 'compile', '--warnings', 'all', '--fqbn', fqbn, projectpath]
-        proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
-                                stderr=subprocess.PIPE)
-        r = proc.wait()
-        out = proc.stdout.read()
-        err = proc.stderr.read()
-        if r == 0:
-            ColorPrint.print_pass(CHECK)
-            if err:
-                # also print out warning message
-                ColorPrint.print_fail(err.decode("utf-8"))
-        else:
-            ColorPrint.print_fail(CROSS)
-            ColorPrint.print_fail(out.decode("utf-8"))
-            ColorPrint.print_fail(err.decode("utf-8"))
-            success = 1
-
 
     for platform in platforms:
         fqbn = ALL_PLATFORMS[platform][0]
         print('#'*80)
         ColorPrint.print_info("SWITCHING TO "+fqbn)
-    install_platform(":".join(fqbn.split(':', 2)[0:2])) # take only first two elements
+        install_platform(":".join(fqbn.split(':', 2)[0:2]), ALL_PLATFORMS[platform]) # take only first two elements
         print('#'*80)
         if not IS_LEARNING_SYS:
-        test_examples_in_folder(BUILD_DIR+"/examples")
+            test_examples_in_folder(platform, BUILD_DIR+"/examples")
         else:
-        test_examples_in_folder(BUILD_DIR)
+            test_examples_in_folder(platform, BUILD_DIR)
+
+
+if __name__ == "__main__":
+    main()
     exit(success)

doxy_gen_and_deploy.sh

@@ -18,6 +18,7 @@ __AUTHOR__="Jeroen de Bruijn, modified by ladyada"
 # Optional global variables:
 # - DOXYFILE            : The Doxygen configuration file.
 # - PRETTYNAME          : A string name of the project (for the doxy headers)
+# - PRESERVE_FOLDERS    : Comma-separated list of folders to preserve during cleanup
 #
 # For information on how to encrypt variables for Travis CI please go to
 # https://docs.travis-ci.com/user/environment-variables/#Encrypted-Variables
@@ -84,6 +85,19 @@ git config --global push.default simple
 git config user.name "Doxygen CI"
 git config user.email "ci-arduino@invalid"
 
+# Check if PRESERVE_FOLDERS is set and back them up
+if [ -n "$PRESERVE_FOLDERS" ]; then
+  echo "Preserving folders: $PRESERVE_FOLDERS"
+  mkdir -p /tmp/preserved
+  # Move preserved folders to temp dir
+  for folder in ${PRESERVE_FOLDERS//,/ }; do
+    if [ -d "$folder" ]; then
+      echo "Backing up folder: $folder"
+      cp -r "$folder" /tmp/preserved/
+    fi
+  done
+fi
+
 # Remove everything currently in the gh-pages branch.
 # GitHub is smart enough to know which files have changed and which files have
 # stayed the same and will only update the changed files. So the gh-pages branch
@@ -99,6 +113,16 @@ else
     rm -r -- !(index.html) || true
 fi
 
+# Restore preserved folders if they were backed up
+if [ -n "$PRESERVE_FOLDERS" ]; then
+  for folder in ${PRESERVE_FOLDERS//,/ }; do
+    if [ -d "/tmp/preserved/$folder" ]; then
+      echo "Restoring folder: $folder"
+      cp -r "/tmp/preserved/$folder" ./
+    fi
+  done
+fi
+
 # Need to create a .nojekyll file to allow filenames starting with an underscore
 # to be seen on the gh-pages site. Therefore creating an empty .nojekyll file.
 # Presumably this is only needed when the SHORT_NAMES option in Doxygen is set
@@ -157,6 +181,8 @@ cd code_docs/${REPO_NAME}
 # both exist. This is a good indication that Doxygen did it's work.
 if [ -d "html" ] && [ -f "html/index.html" ]; then
 
+    case "$GITHUB_REF_NAME" in
+    (main|master)
         echo 'Uploading documentation to the gh-pages branch...'
         # Add everything in this directory (the Doxygen code documentation) to the
         # gh-pages branch.
@@ -180,10 +206,14 @@ if [ -d "html" ] && [ -f "html/index.html" ]; then
           -m "Commit: ${COMMIT_SHA1}"$'\n'"${BUILD_ID}"
 
         # Force push to the remote gh-pages branch.
-    # The ouput is redirected to /dev/null to hide any sensitive credential data
+        # The output is redirected to /dev/null to hide any sensitive credential data
         # that might otherwise be exposed.
         echo 'Git pushing'
         git push --force "https://${AUTH}@github.com/${REPO_SLUG}.git" > /dev/null 2>&1
+        ;;
+    (*)
+        echo 'Not the main branch, not pushing documentation'
+    esac
 else
     echo '' >&2
     echo 'Warning: No documentation (html) files have been found!' >&2

example_actions.yml

@@ -7,26 +7,26 @@ jobs:
     runs-on: ubuntu-latest
     
     steps:
-    - uses: actions/setup-python@v1
+    - uses: actions/setup-python@v4
       with:
         python-version: '3.x'
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v3
       with:
          repository: adafruit/ci-arduino
          path: ci
 
-    - name: pre-install
+    - name: Install the prerequisites
       run: bash ci/actions_install.sh
 
-    - name: test platforms
+    - name: Check for correct code formatting with clang-format
-      run: python3 ci/build_platform.py main_platforms
-
-    - name: clang
       run: python3 ci/run-clang-format.py -e "ci/*" -e "bin/*" -r .
 
-    - name: doxygen
+    - name: Check for correct documentation with doxygen
       env:
         GH_REPO_TOKEN: ${{ secrets.GH_REPO_TOKEN }}
         PRETTYNAME : "Adafruit Arduino Library"
       run: bash ci/doxy_gen_and_deploy.sh
+
+    - name: Test the code on supported platforms
+      run: python3 ci/build_platform.py main_platforms

run-clang-format.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 """A wrapper script around clang-format, suitable for linting multiple files
 and to use for continuous integration.