How to Release a New Version of an Open Source Project Under the Apache Software Foundation

The release process of open-source projects is crucial for ensuring software quality and community collaboration. Take Apache DolphinScheduler as an example, Community Committer Wang Xingjie provides a detailed overview of the open source projects release process under Apache Software Foundation, including environment preparation, process documentation, tool setup, and dependency verification. He also highlights potential issues and solutions encountered during the release process.

1. Release Materials

1. Apache Requirements

   1. Maven Repository Materials (Stored in a temporary repository during the release process)Maven Repository

   2. Release Package, Signature Files, and KeysSign the release version to verify the integrity and authenticity of the downloaded version.

      
      

2. DolphinScheduler Requirements

   1. Dockerhub Images

   2. Helm Files

      
      

3. Permission Requirements

   1. PMC Members have full release permissions.
   2. Committers have limited permissions and require PMC Member assistance.

2. Release Process

Environment Preparation

• JDK: Version 1.8 or higher (1.8.0_271)
• Maven: Preferably version 3.9 or higher (3.9.6)
• Older Maven versions may fail to build deployment packages.
• GPG: All versions (2.0.22)
• SVN: Version 1.6 or higher (1.7.14), lower versions may report errors.

• Git: Version 1.8.3.1
• Server Systems: MacOS, Linux (Linux servers, CentOS7)
• The server must access the Apache website or GitHub. Windows may have issues with syncing GPG and public key IDs; it is recommended to use non-Windows systems if possible.

Basic Tool Preparation

(1) Install GPGFollow the installation documentation.(2) Configure Maven Repository- Modify settings-security.xml- Modify conf/settings.xml (ensure correct path); the Apache LDAP username and password are in plain text.

Dependency Package Verification

Verify the license and download URLs of dependency packages. Check the POM commit logs and review PRs to identify any missing or duplicated dependencies.

Version Modification

1. For convenience in packaging, consider adding environment variables for operations, including SVN tasks.
2. Document Version ModificationNote: docsdev.js references the version-specific files on the Apache DolphinScheduler website. Changes to this file will only take effect after merging the website PR. Ensure to check these files before packaging.

Deployment Package Building and Release

Building the deployment package involves three parts:

1. Release Check

   1. Branch PreparationCreate a prepare branch in the official GitHub repository. Fetch the branch and run the commands. The origin should match the repository you cloned. If you cloned the official repository, use origin; if using a fork, create an upstream pointing to the official repository, and set GH_REMOTE to upstream. All release operations are pushed directly to the official repository. Generally, the prepare branch is based on dev. Use the Git protocol rather than HTTPS to avoid tag push failures.
   2. Release ValidationCommands may require about 8GB of memory. Check the dolphinscheduler-dist/target folder size; if it exceeds 950MB and lacks slimming strategies, pause the release.

2. Release Preparation

Clean up files from the release check by executing the clean command. If some files remain, consider using git stash to temporarily store changes. For multiple releases, clean local and remote tags before running mvn release to avoid errors.

3. Deployment and Release

   a. Execute CommandsThe command results depend on the Release Preparation stage, so do not modify the results (including code). The outcome will appear on Apache Staging Repositories within a few minutes. If files do not appear, check Maven remote repository configuration. If code changes occur during the release, restart the release process from Release Preparation. Use git release:clean and git stash if needed. For multiple releases, monitor file timestamps on Staging Repositories to ensure synchronization.

• b. Submit Release Package to Apache Repository Set SVN_DIR_DEV and SVN_DIR_RELEASE as environment variables if possible. Committers typically do not need to execute this step, as they lack commit permissions. DolphinScheduler currently does not allow this setting, so export the GPG key ID and send it to the PMC for addition, or include it in the release email. Ensure tar.gz.asc files are correctly handled and manually copied if necessary.

Some servers may not have the shasumcommand. You can use sha512sum as an alternative. For example, use sha512suminstead of shasum -a 512.

c. Verify Release FilesCheck additional files, such as doc/conf/docsdev.js. After this, the release packaging work is complete.

Release Emails

1. Create a Release Node using Python 3.0 to generate an MD file with PR links. Verify PR names and locations to avoid errors. After creating the Release Node, a workflow will be triggered to build Docker images and Helm files. Check the GitHub Actions logsfor details.

2. If creating multiple Release Nodes, ensure GitHub Releases do not contain duplicates. Delete old nodes before creating new ones.

3. There are three emails to send:

   a. Release Vote EmailFollow the guidelines in the documentation, and ensure to subscribe to the dev mailing list for visibility. Verify all links before sending. Wait at least 72 hours for PMC votes. If no PMC votes are negative, the voting ends.

   b. Vote Completion EmailFollow the guidelines in the documentation and inquire about PMC titles if necessary.

   c. Final Email

   Details are to be provided in subsequent sections.

Website Documentation Updates

For new versions, update the version information in the Issue Template. Ensure accurate modifications to avoid content loss or accessibility issues on the website. Clear browser cache or use incognito mode if necessary.

After updating, send an email notifying the release completion. Delete the prepared branch after the release email.

News Release

Draft a news release describing the major changes in the version, focusing on bug fixes and improvements. Include key PR details and images if available. Translate PR names for Chinese documents. Use Python 3.0 for final tools and execution.

3. Version Validation

1. Website Documentation

   Log in to the website to verify if the version is displayed correctly.

2. Dockerhub Images

   Check all service images for updates and tags.