PCG logo
Article

Linting and Formatting in AWS CodePipeline

Imagine assembling a jigsaw puzzle. Each piece must fit perfectly to reveal the complete picture. Similarly, in the world of software, linting and formatting ensure every “code piece” fits perfectly together, making the software robust and efficient. Let me guide you through our journey of integrating these processes into a project in AWS CodePipeline.

Why Linting and Formatting are Indispensable

Code Quality is paramount. It’s not just about preventing errors; it’s about ensuring consistency, readability, and efficient collaboration. As teams grow, maintaining this quality becomes challenging. Enter linting and formatting - tools that become our first line of defense against inconsistencies and potential errors.

A Snapshot of Our Development Stack

Before I dive into the details, let’s start with a brief overview of our development stack. We used AWS CodePipeline for the CI/CD pipeline, AWS CodeCommit for the version control system, and AWS CDK for the Infrastructure as Code (IaC). The website was to be deployed with AWS S3 static website hosting, and AWS CloudFront distribution for content delivery.

Architectural Diagram

image-8552e5f4dfe4

Integrating Linting: Where and How?

There are three main stages where linting could be integrated into the development pipeline:

  1. In the Editor: This provides immediate feedback but relies heavily on individual developer setups.
  2. Git Hooks: This ensures standardized checks at some point before code merges.
  3. CI/CD Pipeline: A fail-safe ensuring nothing bypasses our checks.

Exploring Server-Side Hooks

We first considered server-side hooks. These are specialized scripts that spring into action after a code push to the remote repository.

Among the hooks available in this group, our attention was particularly drawn to the Pre-receive hook. This unique hook functions as a gatekeeper, activating right before the code push is accepted. The success of the git push operation hinges on the successful validation of the script; if it validates, the push proceeds, if not, it’s halted right in its tracks.

Below is a visual representation detailing the steps that get triggered when a developer commits to the repository.

image-7427c398b90a

Within GitLab, while server-side hooks provide an extensive range of options, nuances arise in their implementation and maintenance. Comparatively, GitHub’s approach, although simpler, is restricted to their Enterprise tier.

Our AWS-aligned ecosystem had us researching the possibility to implement this in AWS CodeCommit. However, the lack of native support for server-side hooks had us reconsider our strategy.

The Shift to Pre-Commit Hooks

Incorporating linting checks at the pre-commit stage ensures that code is validated even before it’s pushed to the repository. The following chart provides a visual flow of the sequence of steps initiated when pre-commit hooks are triggered.

image-84ae176e8a75

This approach aligns with the principle of catching issues early and ensuring that developers are immediately aware of any corrections or modifications to their code.

Peeking into Our Code Implementation

This is a CDK project using Python, and the linting and formatting guide used by the libraries is the PEP8 standard. We used flake8==4.0.1 for the linting and black==22.3.0 for the formatting.

The Deploy Stage

Let’s start with our deploy stage. This is where we set the AWS environment, specifying the account and region where the stack (which has already been created) will be deployed.

python
Code Copied!copy-button
# Import necessary modules and classes from AWS CDK
from constructs import Construct
from aws_cdk import Stage, Environment
from .MyWebStack import SampleStack

# Define the DeployStage class, which is a subclass of the AWS CDK Stage class
class DeployStage(Stage):
	def __init__(self, scope:Construct, id:str, **kwargs):
		super().__init__(scope, id, **kwargs)
		# Create a new instance of the SampleStack Stack where the infrastructure is defined
		SampleStack(
			self,
			"SampleService",
			# Specify the AWS account and region where the stack will be deployed
			env=Environment(account="12XXXXXXXXXX90", region="eu-central-X")
		)

The Pipeline Stack

Next, let’s delve into the Pipeline Stack. Here, we define our AWS CodePipeline, which includes a reference to our AWS CodeCommit repository and the steps to be executed in the pipeline.

python
Code Copied!copy-button
# Import necessary modules and classes from AWS CDK
from aws_cdk import pipelines, Stack, aws_codecommit as codecommit
from .PipelineStages import DeployStage

# Define the PipelineStack class, which is a subclass of the AWS CDK Stack class
class PipelineStack(Stack):
	def __init__(self, scope: Construct, id: str, **kwargs):
		# Call the constructor of the parent class
		super().__init__(scope, id, **kwargs)
		# Create a reference to an existing AWS CodeCommit repository
		repo = codecommit.Repository.from_repository_name(self, id="TestPipelineRepo", repository_name="test-pipeline")
		# Create a new AWS CodePipeline
		pipeline = pipelines.CodePipeline(
			self,
			id="TestPipeline",
			# Define the steps to be executed in the pipeline
			synth=pipelines.ShellStep(
			id="Synth", 
			# Specify the source of the code that will be used in the pipeline
			input=pipelines.CodePipelineSource.code_commit(repository=repo, branch="main"),
			install_commands=["npm i -g npm@latest"],
			commands=[
				"make install",  # Install necessary dependencies
				"make lint",     # Run linting checks using flake8 . --exclude=*.pyc,.venv/,.pytest_cache/,__pycache__/, cdk.out --max-line-length 80
				"make synth"     # Generate CloudFormation template
			],
			),
		)
		# Create a new instance of the DeployStage class and add it to the pipeline
		deploy = DeployStage(self, id="Deploy")
		pipeline.add_stage(deploy

The Pre-commit Hook

To set up the pre-commit hook, you have to navigate to the hook's directory of your git project. This can be found in ./.git/hooks/. The .git directory is usually hidden by default in case you are wondering why you can’t find it, remember you have to enable ‘view hidden files’ in file browser or run ls -a in your terminal.

In this hook's directory you will see sample hooks that have been bootstrapped with your repo, but if you want them to run you need a copy without the sample extension at the end. Instead of pre-commit sample, you want a file that is just pre-commit. You can set up your pre-commit commands here.

Next, you must navigate to your hook's directory in your terminal (if you were not already using your terminal) and make the pre-commit file executable using this command:

Code Copied!copy-button
chmod +x pre-commit .

There are third party libraries that can help with configuring hooks, but this here is a good option if you need just this one time setup.

Finally, let’s examine our pre-commit hook. This script is run before every local commit in this repository, helping to ensure code quality by automatically formatting the code and adding all changes to the stage for commit.

Code Copied!copy-button
#!/bin/bash

# Run the command to install necessary dependencies
make install

# Run the command to format the code using black .
make fmt

# Stage all changes for commit
git add .

# Echo a message to the console
echo "pre-commit: make fmt"

# End the script with a status of 0 to indicate success
exit 0

Wrapping Up

Integrating linting and formatting into your development workflow is crucial for maintaining code quality. Our journey to integrate linting and formatting into our development pipeline was paved with trials, explorations, and ultimately, insightful revelations. It led us to explore various options, including server-side Git hooks, but we encountered challenges specific to our development environment and proposed collaboration culture. The combination of pre-commit hooks and additional checks in the CI pipeline emerged as the optimal solution.

Have you had a similar experience? Feel free to share your thoughts and questions in the comments below. After all, shared knowledge propels the tech community forward.

Stay Connected

If you have questions, are looking for more insights, or wish to discuss potential collaborations, We’re here to help. Your feedback and inquiries are always welcome.


Services Used

Continue Reading

Article
AWS Lambda: Avoid these common pitfalls

It's a great offering to get results quickly, but like any good tool, it needs to be used correctly.

Learn more
Case Study
Financial Services
Cloud Migration
The VHV Group's Cloud Journey - Strategy for Success

How does an insurance company with more than 4,000 employees balance compliance, modernization, and cost efficiency?

Learn more
Case Study
Financial Services
DevOps
A KYC Archival System for a Digital Bank

Building a KYC archival Cloud platform for a digital bank to store customers’ KYC data.

Learn more
Case Study
Software
DevOps
Accounting Accelerates

What began as a start-up in their parents' basement has developed into a leading provider of cloud-based accounting and financial software within just a few years: sevDesk.

Learn more
See all

Let's work together

United Kingdom
Arrow Down