Back-end
This page outlines the processes used for implementing Continuous Integration and Continuous Deployment (CI/CD) using GitLab CI/CD, Mirroring and a Dockerfile on Render.
GitLab CI/CD
As a result of using a self hosted version of Gitlab, the repository would not integrate well with the Render Gitlab integration. The GitLab CI/CD is used to validate the commits by building and testing the application, before mirroring the repository to our public Github repository, where Render uses the Github repository for deploying the application.
The Gitlab CI/CD consists of two stages:
- Build stage: This stage compiles the application using the mvn compile command.
- Test stage: This stage runs the tests using the mvn test command.
- Deploy stage: This stage prepares the Jacoco code coverage report, installs the application, generates Javadoc, and builds the Maven site. The Jacoco report and the API documentation are then copied to a public directory. This stage only runs if the commit is on the default branch, as specified by the rules section.
If any of these stages fail, the commit will not be mirrored to Github.
Links to the pages generated in the deploy stage:
- Jacoco tests: https://idatt2106-v23-03.pages.stud.idi.ntnu.no/backend/jacoco/index.html
- Javadoc Documentation: https://idatt2106-v23-03.pages.stud.idi.ntnu.no/backend/apidocs/index.html
Mirroring
Gitlab repositories has the functionality to mirror repositories to and from external sources. Smartmat uses the push
mirroring method. Allowing for branches, tags, and commits will be synced automatically
The mirror can be found at https://github.com/Stianlyng/Smartmat
Properties
- On the top bar, select Main menu > Projects and find your project.
- On the left sidebar, select Settings > Repository.
- Expand Mirroring repositories.
Authentication
The mirror uses an ssh://
url, which means we authenticate the push from the Gitlab instance using public key authentication. Generating a deploy key in the Github repository can be done by:
- On the top bar of the repository, select Settings > Deploy keys.
- Clicking Add deploy key creates a new key.
- Copying the public key, and later adding it to the verify host key box.
Deployment using Render
Render uses the Dockerfile to build and deploy the application. If the GitLab CI/CD pipeline and mirroring to github is successful, the Dockerfile is used to create a Docker image and deploy the application.
The Dockerfile consists of two stages:
-
Build stage: The base image uses the
eclipse-temurin:19-jdk-alpine
image. It sets the working directory, installs maven, and add the dependencies defined in thepom.xml
file. At last it packages the application using maven. -
Runtime stage: This stage uses the same
alpine
image, as a new base image, for creating a smaller container. It copies the compiled JAR file, and exposes the application port.
Deployment
The deployment can be set as both manual and automatic. The application currently uses the manual option, using a deployment hook for triggering a new deployment.
Front-end
GitLab CI/CD
The CI/CD utilizes the node:16-buster
Docker-Image.
The front-end GitLab CI/CD pipeline automates the testing and deployment process for the project. It consists of five essential stages:
-
Install: Installs all necessary dependencies using
npm ci
for reliable environment installation. -
Build: Compiles the project into a production-ready version using npm run build.
-
Test: Executes unit tests with Vitest, a fast and lightweight test runner. Generates text report artifact.
-
Security: Uses OWASP Dependency Check to scan for security vulnerabilities in the project dependencies. Generates json report artifact.
-
Deploy: Deploys the built code to GitHub Pages by creating a CNAME, and a 404.html file, initializing a git repository, committing the change. Then the stage pushes to the GitHub Pages repo.
Examples of the generated reports are found at the bottom of this page.
Github-Pages
Github-pages hosts the front-end. This is a static site hosting service that takes files from a GitHub repository, optionally processes them, and publishes a website.
Deployment
The deployment stage consists of these essential steps:
- Creates a CNAME file with the domain name for the project. Initializes a git repository.
- Adds and commits the changes with the message "Auto-deploy to GitHub Pages".
- Sets the remote origin to the GitHub Pages repository, using a secure method to handle the
$GITHUB_TOKEN
. - Pushes the changes to the GitHub Pages repo: https://github.com/JFBendvold/JFBendvold.github.io
The result for the Github Pages repo:
Frontend testing
Unit testing with Vitest
The frontend of SmartMat has been developed and verified with unit tests. The framework chosen is Vitest. The focus of the unit tests is the business-critical part of the code, being the Vue-component-files. The file coverage for the unit tests are estimated at 57% of the vue-components.
End to end testing with Cypress
The user stories of the application is tested by using integration testing in Cypress. The coverage for the essential user stories is about 50%. The Cypress-tests also verifies the urls of the website.
Screenshot of running Cypress-tests for the final product:
Test- and security reports:
Example of the file generated by the CI-unit testing stage: clean_test_output.txt
> smartmat@0.0.0 test:unit
> vitest
RUN v0.29.8 /builds/idatt2106-v23-03/frontend/SmartMat
✓ src/components/__tests__/setup/SetupWelcome.spec.js (4 tests) 148ms
✓ src/components/__tests__/setup/SetupComplete.spec.js (3 tests) 79ms
stdout | unknown test
Dette er en eksempelfeilmelding
✓ src/components/__tests__/Welcome.spec.js (1 test) 66ms
✓ src/components/__tests__/home/Welcome.spec.js (4 tests) 33ms
✓ src/components/__tests__/ErrorPopup.spec.js (2 tests) 33ms
✓ src/components/__tests__/list/ListHeader.spec.js (3 tests) 94ms
✓ src/components/__tests__/Footer.spec.js (5 tests) 146ms
✓ src/components/__tests__/home/InfoSection.spec.js (4 tests) 87ms
✓ src/components/__tests__/statistics/LevelProgress.spec.js (3 tests) 112ms
✓ src/components/__tests__/navbar/Navbar.spec.js (4 tests) 184ms
✓ src/components/__tests__/statistics/ColumnComparison.spec.js (2 tests) 106ms
✓ src/components/__tests__/XpPopup.spec.js (1 test) 198ms
✓ src/components/__tests__/InfoButton.spec.js (3 tests) 273ms
✓ src/components/__tests__/setup/SetupAllergies.spec.js (4 tests) 174ms
stdout | src/components/__tests__/statistics/ProgressChart.spec.js > ProgressChart component renders > renders the ProgressChart component
<div class="column-comparison">
<apexchart width="300" height="300" type="line" options="[object Object]" series="[object Object],[object Object]"></apexchart><svg class="svg-inline--fa fa-circle-info info-button" aria-hidden="true" focusable="false" data-prefix="fas" data-icon="circle-info" role="img" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<path class="" fill="currentColor" d="M256 512A256 256 0 1 0 256 0a256 256 0 1 0 0 512zM216 336h24V272H216c-13.3 0-24-10.7-24-24s10.7-24 24-24h48c13.3 0 24 10.7 24 24v88h8c13.3 0 24 10.7 24 24s-10.7 24-24 24H216c-13.3 0-24-10.7-24-24s10.7-24 24-24zm40-208a32 32 0 1 1 0 64 32 32 0 1 1 0-64z"></path>
</svg>
<!--v-if-->
</div>
✓ src/components/__tests__/statistics/ProgressChart.spec.js (3 tests) 291ms
✓ src/components/__tests__/statistics/WastePieChart.spec.js (3 tests) 295ms
✓ src/components/__tests__/NavFooter.spec.js (1 test) 209ms
✓ src/components/__tests__/login/Register.spec.js (5 tests) 43ms
✓ src/components/__tests__/settings/ProfileSettings.spec.js (1 test) 87ms
✓ src/components/__tests__/settings/AllergensSettings.spec.js (1 test) 249ms
✓ src/components/__tests__/statistics/Statistics.spec.js (1 test) 204ms
Test Files 21 passed (21)
Tests 58 passed (58)
Start at 13:30:06
Duration 8.20s (transform 7.57s, setup 2ms, collect 29.91s, tests 3.11s)
Example of the file generated by the CI-OWASP dependency check stage: owasp_dependency_report
These document-artifacts are also found by downloading from pipeline-runs: frontend@e54bb0e3
The json file is found under downloadable artifacts for the security stage:
Backend testing
Testing when committing new changes
When a new change is committed the ci/cd will run tests automatically to check the integrity of the change. If the commit not meant for the main branch, then the jacoco test report will be skipped. On the other hand when a new merge request for main is pushed, not only will it run all tests, but also create a new jacoco test report and send it to gitlab pages. Merge requests are only allowed when all tests succeed, or if they are force merged.
A commit to a non-main branch that failed:
A test failure in the pipeline:
Unit testing with junit5
We used unit tests to verify the integrity of the backend components. Junit 5 is integrated with spring boot, and is a fast and easy way to test components in java. All vital classes and methods where tested, and the test coverage in jacoco was 75%, while the test coverage in intellij was 98%.