Components Publication

Workflow to publish open source components

Regular Git Repository Update

  1. Push the changes to the GitHub repository Open Source Components.

    1. Verify that the build job in the CI pipeline was successful.

  2. The continuous integration pipeline is triggered by the push to the GitHub repository. Check that the job has been completed successfully. See Readme.

The statistics for developer activities will be automatically updated.

A good sanity check before pushing to the repository is to run the following command before pushing the code to the repository:

./gradlew clean build test javadoc testCodeCoverageReport testAggregateTestReport -Pvaadin.productionMode

You can consult the various reports generated by opening <PROJECT_ROOT>/src/site/reports.html in your browser.

Release Process

We use a monorepo to manage the source code of the libraries and the example application. All libraries are released and published together on the maven central repository.

  1. push to GitHub and verify the build on the CI server

  2. git commit -m "Release X.Y.Z"

  3. git tag -a "tangly-os-X.Y.Z" -m "official release published on maven central"

  4. git push origin --tags

  5. ./gradlew publishMavenJavaPublicationToMavenRepository -Pvaadin.productionMode

Run all build-related tasks with -Pvaadin.productionMode due to a bug in the vaadin plugin.

Verify first that the example application runs correctly with the new artifacts.

  1. Publish artifacts to the local maven repository ./gradlew publishMavenJavaPublicationToMavenLocal. The example application uses the artifacts from the local maven repository to compile and run.

  2. Update the tangly App Example. Update the artifacts in the build.gradle file to the latest version.

  3. Verify the application runs correctly with the new artifacts.

Upon releasing the new version of the libraries, you have to wait until the artifacts are available on the maven central repository. Once the artifacts are available, you can push the new version of the example application to the GitHub repository. It triggers the CI/CD pipeline.

You can check the release on the maven central repository under Maven Central net.tangly. The search index takes a few hours to be updated.

Log in the sonatype portal for the staging repository and close the repository. Search the component in

Constraints

The build process of the Gradle build file requires these variables to run without errors.

You must create a gradle ~/.gradle/gradle.properties file with the following content:

mavenCentralUsername=*Value*
mavenCentralPassword=*Value*

signing.secretKeyRingFile=*Path to your secring.kbx*
signing.keyId=743B11D9
signing.password=*Password to unlock the secret key*

For mysterious reasons, the Maven Publish plugin does work only if you have a secring.kbx file. If necessary, run the command gpg --keyring secring.gpg --export-secret-keys > ~/.gnupg/secring.gpg to create the file [1].

Maven Central Publication

The following process is used to publish tangly open source component on Maven Central.

  1. Publish artifacts of a component with the script uploadToMavenCentral.sh. Do not use the gradle task gradle publishAllPublicationsToMavenRepository -Pvaadin.productionMode. Sonatype cannot handle a parallel upload of multiple components and simply close the connections during upload (Status 2020-Q2).

    1. Pre-activity: test publishing with gradle publishToMavenLocal.

    2. Pre-activity: check on GitHub the product compiles through the CI continuous integration pipeline.

    3. Post-activity: tag repository on git to identify the version of the published packages. The tag version semantic part must be the same as the package version on Maven Central.
      The tag is of the form -AcronymComponent-MajorVersion-MinorVersion-PatchVersion-.

    4. Verify with git tag label and git push origin --tags.

  2. Log into Sonatype Maven Central Repository.

    1. Go to Staging Repositories.

    2. Find your temporary repository at the end of the list.

    3. Review the content of the repository,

    4. Close the repository-button on the tool list.

    5. Close stage is initiated and checks are performed. If everything was fine, the content was published.

    6. Release the artifact-button on the tool list.

  3. See Static Website Publication to update the website.

It takes up to one day until the component is visible in Maven Central website upon publication.

On the positive side, the last publications were performed in less than 30 minutes.

The final step is to create a release on GitHub and update the website.

Dependencies

Below the dependencies between tangly open source components as defined in the multi-modules Gradle Build Tool build file.

nettanglyerpbddcommonscoredevfsmgleamuicollaboratorscrminvoicesledgerproductsui

Decisions

  1. The open source components are stored in a mono repository. Each component is declared as a Gradle module.

  2. We use de facto standard Git.

  3. We are hosted at GitHub [2]. Mirror repositories are hosted at GitLab [3].

  4. The continuous delivery pipelines work for the primary and the mirror platforms. The build uses the eclipse-temurin Java distribution. This docker image is based on Ubuntu. The installation of additional packages is therefore simplified [4].

The synchronization is performed with the command

git push --prune
      https://github.com/marcelbaumann/tangly-os.git
      +refs/remotes/origin/*:refs/heads/* +refs/tags/*:refs/tags/*        (1)

git remote                                                                (2)

git push origin                                                           (3)
git push origin-gitlab
git push origin-hslu
git push origin-bitbucket
1 Push the repository with all remotes, references, and tags to GitHub specified repository
2 List all remotes associated with this git repository
3 Push the master branch to the origin-GitHub remote repository. The push shall trigger the continuous integration pipeline

You must create a token for a developer account to be able to push changes and trigger the CI pipeline. The username is your account username. The password is the generated token [5]. Store it in a safe place.

Findings

  1. We needed to manually publish the public key on keys.openpgp.org

  2. Gradle build file has a condition including disabling the signing plugin on the CI pipeline because keys and associated authentication are not available. I decided not to upload private keys to bitbucket. See the documentation at the beginning of the Gradle build file for details.

Set the property if you want to sign the component artifacts and publish to Maven Central.

Static Website Publication

Introduction

The static website uses Asciidoc to write all documents.

The static website uses Hugo as site generator. The Hugo theme is Docsy.

Configure Asciidoctor

The new version of the Hugo support configuration of Asciidoctor through config.toml therefore no manipulation of asciidoctor is needed. As a bonus, the files generated by PlantUml are created in the correct folder. We still need to install the referenced packages.

sudo gem install asciidoctor-diagram (1)
sudo gem install asciidoctor-bibtex (2)
sudo gem install asciimath (3)
1 Provides support for all diagrams such as plantUML and mermaid.
2 Provides support for formal bibliography references.
3 Provides support for mathematical and logical expressions in documents.

Docsy Tailoring

The docsy theme is missing some features. It does not support AsciiDoc styling or commenting blogs. Our extensions follow the official rules on how a theme can be tailored and expanded.

The major changes are:

  • Inclusion of the styling sheet asciidoctor.scss to style AsciiDoc documents [6].

  • Inclusion of the shortcodes extension shortcodes.html layouts and the style file shortcodes.scss to support file attachments in the website [7].

  • Extension of the blog section with comments-uttereances.html to support comments on blog articles [8].

  • Changes in the header and footer partials to display mathematical expressions and well-styled copyright [9].

All changes are defined in the assets and layouts folders as part of our website content. Hugo’s algorithms select local overwritten files and ignore the corresponding theme files.\

The current structure of our extensions follows the official approach to how a Hugo module should be extended. We do not change any files in the theme. All changes are defined locally. We use the selection algorithm of Hugo to activate our tailoring and extensions.

Create Website

The tangly open source components website is hosted under Documentation.

Read the instructions in the asciidoctor script file under the scripts' folder. Four scripts are provided to generate the Hugo static site with associated structure and theme.

  1. the script initiateHugo.sh creates the layout of the site and retrieves the theme.

  2. the script populateHugo.sh populates the site with our content.

Upon completion of local development, you must start a local Hugo server and generate the indexing files for lunr search. Before uploading the site, stop the Hugo local server.

  1. the script completeHugo.sh generates the static pictures and copies them to the static folder and publish the whole site on GitHub.

Upon completion, the site is published on the web for all.

Set JDK Version under macOS

Install regular Java JDK such as the Oracle or OpenJDK distribution. They will be located under /Library/Java/JavaVirtualMachines.

If using bash or zsh shell, add the following commands to your .bash_profile or .zshrc [10].

alias java21 = "export JAVA_HOME=`/usr/libexec/java_home -v 21`; java -version"
alias java23 = "export JAVA_HOME=`/usr/libexec/java_home -v 23`; java -version"

You can set the exact version such as 17.0.1 to precisely select a JDK or set an overall version such as 17 to select a generic version.

After restarting your terminal, the command java17 will set JDK 17 to default JDK (if installed on your macOS).

The list of JDK can be found with

/usr/libexec/java_home -V

Create Docker Distribution

Generate the ERP application as a production release.

gradle install -DproductionMode=true (1)
gradle install -Pproduction
unzip ./build/distributions/net.tangly.erp.ui-<version>.zip (2)
docker build -t erp-docker . (3)
docker run -ti -p 8080:8080 erp-docker (4)
1 Generates a production distribution stored under ./build/distributions
2 Unzip the packed distribution because the docker base image does not contain the unzip utility
3 Build the docker image named erp-docker
4 Run the docker image erp-docker and map the image port to 8080
1. The newer version macOS GPP stopped creating the secring.kbx file and stored public and private keys into the pubring.kbx file.
2. We moved from Bitbucket to GitHub in 2022. The reasons were that the new platform supports publishing of static websites with a lot more functionalities. The new agile project management functionality with Scrum boards, sprint iterations, and insight report is riveting.
3. The support for the mirrors Bitbucket and the technical university of Lucerne was removed due to obsolete technology and lack of interest.
4. It is a shame that each hosting platform has created their own incompatible pipeline script syntax.
5. This is necessary if you have enabled two factors authentication. The other approach is to generate an SSH key and perform all operations other SSH. This is my preferred approach.
6. I asked for a change request on GitHub for Docsy to better support AsciiDoc documents. The Docsy team decided that their priorities are different and rejected the request.
7. This change is defined as a merge request. The pull request is quite old and seems to have a low priority.
8. The current setup of the template does not allow selection of a comment solution per configuration.
9. An issue report exists on GitHub about the copyright issue.
10. zsh is the default shell in current macOS versions.