Morioh
Login
Federico Laresi

Federico Laresi

5 years ago

How to Using Cypress with Django and Vue for integration testing in GitLab CI

This article provides a high level description of my attempts at using Cypress for integration testing in an a Django + VueJS app using GitLab CI. Here is the GitLab repo that I will be referencing and copying code samples from below.

I have recently been working on CI/CD pipelines using GitLab CI for a project that uses Django REST Framework, Celery, Celery Beat and Django Channels for the backend with a separate static frontend site made with Quasar, a fantastic framework and component library for Vue. Here's an overview of the stages in my pipeline:

  • Documentation: Deploy a static VuePress documentation site to GitLab pages under my GitLab group.
  • Test: Linting and unit testing for python and javascript using flake8, pytest, eslint and jest.
  • Build: Build the static assets for the Vue frontend as well as the image for the backend application once they have passed all tests in the teststage. This uses a multi-stage Dockerfile discussed later on.
  • Integration: Run integration tests using the static files, the backend docker image, postgres and redis. This uses Cypress to run headlessly, but the videos of each test are recorded and stored in GitLab as job artifacts.
  • Release: Tag the image with the commit SHA and push it from GitLab's registry to the production registry (Elastic Container Registry or ECR)
  • Deploy to staging: Sync static files from the build stage to an S3 bucket that is served on a CloudFront distribution, and update the CloudFormation stack with the commit SHA to trigger ECS to make a rolling update that will use the new docker image in the different services that run in the ECS cluster (django, celery, beat and channels). Optionally run database migration tasks in ECS if there are changes in the migrations folders, and run collectstatic if there are changes in the static directories.
  • Deploy to production (not implemented yet): Same as the previous stage, but applied to the production CloudFormation stack.

First steps with Cypress

Cypress allows you to easily mock server calls with cy.server(). My first attempt at using Cypress mocked all backend calls and only tested the Vue app. This approach might be sufficient if you have a simple backend which is well tested. Here's what the GitLab CI job looked like:

.test e2e:
  image: cypress/base:10
  stage: test
  script:
    - cd frontend
    - npm install
    - apt install httping
    - npm run serve &
    - while ! httping -qc1 http://localhost:8080/login ; do sleep 1 ; done
    - $(npm bin)/cypress run

This makes use of the official Cypress base image which includes all dependencies. npm run serve & starts the development server in the background and then waits for it to be available with httping before starting the tests.

Using docker-compose with docker-in-docker

One popular approach to integration testing uses docker-compose. Here's an example from testdriven.io that is use in a Flask/React app:

# run e2e tests
e2e() {
  docker-compose -f docker-compose-stage.yml up -d --build
  docker-compose -f docker-compose-stage.yml exec users python manage.py recreate_db
  ./node_modules/.bin/cypress run --config baseUrl=http://localhost --env REACT_APP_API_GATEWAY_URL=$REACT_APP_API_GATEWAY_URL,LOAD_BALANCER_DNS_NAME=$LOAD_BALANCER_DNS_NAME
  inspect $? e2e
  docker-compose -f docker-compose-$1.yml down
}

The approach here is to:

  1. Start services
  2. Seed databases
  3. Run cypress tests against the docker-compose stack

This approach allows us to separate each part of our application into its own container. It also allows us to easily run our integration tests locally using docker-compose.

I adapted something similar to this approach. Here's how I put it together. First, here's the GitLab CI job definition:

e2e cypress tests with docker-compose:
  stage: integration
  image: docker:stable
  variables:
    DOCKER_HOST: tcp://docker:2375
    DOCKER_DRIVER: overlay2
  services:
    - docker:dind
  before_script:
    - apk add --update py-pip
    - pip install docker-compose~=1.23.0
  script:
    - sh integration-tests.sh
  artifacts:
    paths:
      - cypress/videos/
      - tests/screenshots/
    expire_in: 7 days

There is a lot of setup in this job definition, but the script stage is where everything happens. Here is integration-tests.sh:

#!/bin/bash

set -e


echo “Starting services”

docker-compose -f docker-compose.ci.yml up -d --build


echo “Running tests”

docker-compose -f docker-compose.ci.yml -f cypress.yml up --exit-code-from cypress


echo “Tests passed. Stopping docker compose…”

docker-compose -f docker-compose.ci.yml -f cypress.yml down

Using –exit-code-from is a useful flag that allows us to run Cypress in a separate container defined in a separate docker-compose file, and exit from the docker-compose command based on the exit code from the cypresscontainer, which should be 0 if the tests pass successfully. If Cypress fails, this script will exit with a non-zero exit code because of set -e.

Here’s the cypress.yml file:

version: ‘3.7’

services:

cypress:

image: “cypress/included:3.4.0”

container_name: cypress

networks:

- main

depends_on:

- nginx

environment:

- CYPRESS_baseUrl=http://nginx

working_dir: /e2e

volumes:

- ./:/e2e

The cypress/included:3.4.0 image already has Cypress installed, and it’s default command is to run Cypress, so we don’t need to define command.

We use http://nginx as the baseUrl for Cypress because we are reaching out to the nginx container which serves our Vue application. The nginx app then reaches out the the backend container by making requests to http://backend.

Here’s the docker-compose.ci.yml file:

version: ‘3.7’

services:

postgres:

container_name: postgres

image: postgres

networks:

- main

volumes:

- pg-data:/var/lib/postgresql/data


backend: &backend

container_name: backend

build:

context: ./backend

dockerfile: scripts/prod/Dockerfile

command: /start_ci.sh

networks:

- main

volumes:

- ./backend:/code

- django-static:/code/static

depends_on:

- postgres

environment:

- SECRET_KEY=‘secret’

- DEBUG=True

- DJANGO_SETTINGS_MODULE=backend.settings.gitlab-ci


asgiserver:

<<: *backend

container_name: asgiserver

entrypoint: /start_asgi.sh

volumes:

- ./backend:/code


nginx:

container_name: nginx

build:

context: .

dockerfile: nginx/ci/Dockerfile

ports:

- 80:80

networks:

- main

volumes:

- django-static:/usr/src/app/static

depends_on:

- backend


redis:

image: redis:alpine

container_name: redis

volumes:

- redis-data:/data

networks:

- main


volumes:

django-static:

portainer-data:

pg-data:

redis-data:


networks:

main:

driver: bridge

We don’t actually need to run asgiserver and backend as separate containers, but I wanted to test this way because it closely resembles the setup I plan to use in production. daphne, the server started in the asgiserver container, is capable of serving regular http requests

GitLab Services

This allowed me to run tests locally by simply running ./integration-tests.sh. While everything passed locally, the websocket test didn’t pass in GitLab CI despite lots of debugging, manual waits in Cypress and other efforts. While this might work for most cases, I was interested in finding another solution that would not use docker-in-docker (dind), or docker-compose.

GitLab has a services feature that allows you to define containers to run in the CI job that can be accessed by the main container. For example, a redisservice can be accessed by redis://redis:6379/0 inside the main container, similar to how networking works in docker-compose. Here’s the GitLab job I defined to try to use a similar approach to the docker-compose setup, but without using docker-compose in favor of services:

.e2e: &e2e

image: cypress/base:8

stage: integration

variables:

# variables passed as env vars to all services

SECRET_KEY: ‘secret’

DEBUG: ‘’

DJANGO_SETTINGS_MODULE: ‘backend.settings.gitlab-ci’

CELERY_TASK_ALWAYS_EAGER: ‘True’

services:

- name: postgres

- name: $CI_REGISTRY_IMAGE/backend:latest

alias: backend

command: [“/start_ci.sh”]

- name: redis

- name: $CI_REGISTRY_IMAGE/frontend:latest

alias: frontend

before_script:

- npm install --save-dev cypress

- $(npm bin)/cypress verify

script:

- $(npm bin)/cypress run --config baseUrl=http://frontend

after_script:

- echo “Cypress tests complete”

artifacts:

paths:

- cypress/videos/

- cypress/screenshots/

expire_in: 7 days

This doesn’t work. After lots of debugging and raising issues in Cypress and GitLab, I came across this merge request and found other users up against the same issue. The issue with this approach is that services are not available to other services defined in a GitLab CI job. If they are in a future release, something like this might work, but for now I’ll need another way.

At this point I started to search for other projects that do Cypress testing on GitLab. Gitter is a cool example. It’s a company that GitLab purchased, and it is open source. Here is the e2e CI job that inspired my next attempt at e2e cypress testing:

.test_e2e_job: &test_e2e_job

<<: *test_job

variables:

<<: *test_variables

ENABLE_FIXTURE_ENDPOINTS: 1

DISABLE_GITHUB_API: 1

NODE_ENV: test-docker

script:

# Cypress dependencies https://docs.cypress.io/guides/guides/continuous-integration.html#Dependencies

- apt-get update -q -y

- apt-get --yes install xvfb libgtk2.0-0 libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2

# Create output/assets/js/vue-ssr-server-bundle.json

- npm run task-js

# Start the server and wait for it to come up

- mkdir -p logs

- npm start > logs/server-output.txt 2>&1 & node test/e2e/support/wait-for-server.js http://localhost:5000

# Run the tests

- npm run cypress – run --env baseUrl=http://localhost:5000,apiBaseUrl=http://localhost:5000/api

artifacts:

when: always

paths:

- logs

- test/e2e/videos

- test/e2e/screenshots

- cypress/logs

expire_in: 1 day

retry: 2

Here’s the *test_job part:

.test_job: &test_job

<<: *node_job

variables:

<<: *test_variables

stage: build_unit_test

services:

- name: registry.gitlab.com/gitlab-org/gitter/webapp/mongo:latest

alias: mongo

- name: redis:3.0.3

alias: redis

- name: registry.gitlab.com/gitlab-org/gitter/webapp/elasticsearch:latest

alias: elasticsearch

- name: neo4j:2.3

alias: neo4j

script:

- make ci-test

Let’s also take a look at *node_job:

.node_job: &node_job

image: registry.gitlab.com/gitlab-org/gitter/webapp

before_script:

- node --version

- npm --version

- npm config set prefer-offline true

- npm config set cache /npm_cache

- mv /app/node_modules ./node_modules

- npm install

artifacts:

expire_in: 31d

when: always

paths:

- /npm_cache/

- npm_cache/

There’s a lot going on in this CI job. If you haven’t used YAML anchors before, the idea is that <<: *job let’s us reference keys that have job: &job. See this article for more information on YAML anchors. Let’s merge the anchors into one key for readability:

.test_e2e_job: &test_e2e_job

image: registry.gitlab.com/gitlab-org/gitter/webapp

before_script:

- node --version

- npm --version

- npm config set prefer-offline true

- npm config set cache /npm_cache

- mv /app/node_modules ./node_modules

- npm install

variables:

<<: *test_variables

ENABLE_FIXTURE_ENDPOINTS: 1

DISABLE_GITHUB_API: 1

NODE_ENV: test-docker

services:

- name: registry.gitlab.com/gitlab-org/gitter/webapp/mongo:latest

alias: mongo

- name: redis:3.0.3

alias: redis

- name: registry.gitlab.com/gitlab-org/gitter/webapp/elasticsearch:latest

alias: elasticsearch

- name: neo4j:2.3

alias: neo4j

script:

# Cypress dependencies https://docs.cypress.io/guides/guides/continuous-integration.html#Dependencies

- apt-get update -q -y

- apt-get --yes install xvfb libgtk2.0-0 libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2

# Create output/assets/js/vue-ssr-server-bundle.json

- npm run task-js

# Start the server and wait for it to come up

- mkdir -p logs

- npm start > logs/server-output.txt 2>&1 & node test/e2e/support/wait-for-server.js http://localhost:5000

# Run the tests

- npm run cypress – run --env baseUrl=http://localhost:5000,apiBaseUrl=http://localhost:5000/api

artifacts:

when: always

paths:

- logs

- test/e2e/videos

- test/e2e/screenshots

- cypress/logs

expire_in: 1 day

retry: 2

Here are some import points to mention about this job:

  • This job starts with a base image of the main webapp container (an express application).
  • Supporting services are defined in the services section: mongo, redis, elasticsearch and neo4j. But there is no communication between these services; there is only communication between the webappcontainer and the individual services.
  • Instead of starting from a Cypress image, Cypress and its dependencies are installed in the container in the script section. Cypress is installed in devDependencies in package.json.
  • The job is set to retry two times. Sometimes e2e tests can be flaky. I have definitely noticed this in my experience with Cypress.

Now let’s take a look at my approach that I adopted from this example. There are two main parts: the GitLab CI job, and the multi-stage Dockerfile. I need to serve the backend Django application and the Vue frontend out of the same container, even though these services are separate in production. This is a perfect use case for a multi-stage Dockerfile. Here’s an overview of the stages in my Dockerfile:

  1. Build the static assets
  2. Build the production backend docker image
  3. Starting FROM the production image, COPY the Vue application into the static folder and install Cypress dependencies.

Here’s the Dockerfile:

# build stage that generates quasar assets

FROM node:10-alpine as build-stage

ENV HTTP_PROTOCOL http

ENV WS_PROTOCOL ws

ENV DOMAIN_NAME localhost:9000

WORKDIR /app/

COPY quasar/package.json /app/

RUN npm cache verify

RUN npm install -g @quasar/cli

RUN npm install --progress=false

COPY quasar /app/

RUN quasar build -m pwa


this image is tagged and pushed to the production registry (such as ECR)


FROM python:3.7 as production

ENV PYTHONUNBUFFERED 1

ENV PYTHONDONTWRITEBYTECODE 1

RUN mkdir /code

WORKDIR /code

COPY backend/requirements/base.txt /code/requirements/

RUN python3 -m pip install --upgrade pip

RUN pip install -r requirements/base.txt

COPY backend/scripts/prod/start_prod.sh 

backend/scripts/dev/start_ci.sh 

backend/scripts/dev/start_asgi.sh 

/

ADD backend /code/


this stage is used for integration testing


FROM production as gitlab-ci


update and install nodejs


COPY --from=build-stage /app/dist/pwa/index.html /code/templates/

COPY --from=build-stage /app/dist/pwa /static

COPY cypress.json /code

RUN mkdir /code/cypress

COPY cypress/ /code/cypress/

RUN apt-get -qq update && apt-get -y install nodejs npm

RUN node -v

RUN npm -v


cypress dependencies


RUN apt-get -qq install -y xvfb 

libgtk-3-dev 

libnotify-dev 

libgconf-2-4 

libnss3 

libxss1 

libasound2

Now let’s look at the GitLab CI job:

e2e: &cypress

stage: integration

image: $CI_REGISTRY_IMAGE/backend:latest

services:

- postgres:latest

- redis:latest

variables:

SECRET_KEY: ‘secret’

DEBUG: ‘True’

CELERY_TASK_ALWAYS_EAGER: ‘True’

before_script:

- python backend/manage.py migrate

- python backend/manage.py create_default_user

- cp /static/index.html backend/templates/

- /start_asgi.sh &

script:

- npm install cypress

- cp cypress.json backend/

- cp -r cypress/ backend/cypress

- cd backend

- $(npm bin)/cypress run

artifacts:

paths:

- backend/cypress/videos/

- backend/cypress/screenshots/

expire_in: 7 days

This jobs starts from the backend:latest image created by the Dockerfile above. It references postgres and redis services. The before_script runs a database migration, seeds the database with a user, and copies index.htmlto the templates folder. Finally, we run start_asgi.sh & in the background. Next, we install cypress and run the tests.

Instead of using different containers for the Django backend, celery and daphne services (the Django Channels ASGI server), we can use only daphne to serve both HTTP and Websocket traffic, and we can set CELERY_TASK_ALWAYS_EAGER to True so that celery tasks are run synchronously in our e2e tests. We can add the following to our urls.py to serve index.html and other static files to serve the Vue application from our Django container:

if settings.DEBUG:

import debug_toolbar # noqa

urlpatterns = urlpatterns + [

path(‘’, index_view, name=‘index’),

path(‘admin/debug/’, include(debug_toolbar.urls)),

# catch all rule so that we can navigate to

# routes in vue app other than “/”

re_path(r’^(?!js)(?!css)(?!statics)(?!fonts)(?!service-worker.js)(?!manifest.json)(?!precache).*', index_view, name=‘index’) # noqa

] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

We also set STATIC_ROOT to /, and disable CsrfViewMiddleware for simplicity. These settings can be found in gitlab-ci.py.

Here’s the simple index_view that serves the index.html file on requests to /, or any other path that does not start with anything in our STATIC folder. This can be found in core/views.py:

# Serve Vue Application via template for GitLab CI

index_view = never_cache(TemplateView.as_view(template_name=‘index.html’))

One feature of GitLab CI I really like is GitLab runner. It is another open-source project that allows us to run GitLab CI jobs locally in the same way they run when you push your code to gitlab.com and trigger a job on a public runner. This is really useful for when you are debugging a CI job locally and don’t want to keep pushing code to gitlab.com to run the pipeline.

In the last part of this article I wan’t to describe how we can test the GitLab CI job locally using GitLab runner.

There is really one change we need to make in order to run this job locally. Let’s define a new job that uses the anchor for the existing job, but overwrite the image key:

# use this test with gitlab-runner locally

e2e-local:

<<: *cypress

image: localhost:5000/backend:latest

In my repo, this job is commented by placing a period in front of the job name (.e2e-local). I don’t want to ever run this job in production, so I need to uncomment the job when running locally and then recomment it when I want to push code to GitLab.

There are just a few steps need to test locally: setup a local registry, build the image, tag the image, and push it to the local registry. Here’s how to do that. Run the following command (taken from docker documentation):

docker run -d -p 5000:5000 --restart=always --name registry registry:2

To build the production image that we will use in the test, run the following command:

docker-compose -f compose/test.yml build backend

Then tag the image with the following command:

docker tag compose_backend:latest localhost:5000/backend:latest

Then push the tagged image to the local registry:

docker push localhost:5000/backend:latest

Finally, commit any current changes you have made. Gitlab runner requires that you commit changes before running tests. Run the GitLab CI job with the following command:

gitlab-runner exec docker e2e-local

Conclusion

That’s a quick tour of the CI/CD pipeline I’m working on with a close look at the integration stage. I would be very interested to hear about anyone else’s way of going integration testing in a project with a similar tech stack. If you have any suggestions for how I could improve the way I’m doing integration tests, I would love to hear your thoughts! Thanks for reading.

Originally published on about.gitlab.com

#vue-js #django #web-development

How to Using Cypress with Django and Vue for integration testing in GitLab CI
95.05 GEEK