Python Library Development

collinmutembei,•micro-serviceslibraryclean-code
Back

Introduction

Microservices are a popular architectural pattern for building modern, scalable and modular applications. In a microservice architecture, services communicate with each other through APIs using a well-defined contract. This contract is often expressed in terms of data schemas and operations on data. In this blog, I will describe how to develop a shared library in Python for sharing common data schemas and operations on data for a microservice architecture. This shared library will enable the different microservices to use the same data schema and data operations, promoting code reuse and consistency.

Designing the Shared Library

The first step in developing a shared library is to design the data schema and operations that will be used by the different microservices. The data schema should be designed to be extensible and easily modifiable. The operations should be designed to be efficient and easily reusable. Some examples of data schema and operations that can be shared in a microservice architecture include:

Data schema:

from pydantic import BaseModel, validator
 
class User(BaseModel):
    id: int
    username: str
    password: str
 
    @validator('username')
    def no_special_chars(cls, v):
        if any(char in "!@#$%^&*()-+_=~`[]{}|;:<>,.?/" for char in v):
            raise ValueError('special characters are not allowed')
        return v

Operations on data:

def hash_password(password: str) -> str:
    return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
 
def verify_password(password: str, hashed_password: str) -> bool:
    return bcrypt.checkpw(password.encode(), hashed_password.encode())

Developing the Shared Library

Once the data schema and operations have been designed, the shared library can be developed in Python. The library can be developed as a Python package, with each data schema and operation implemented as a separate module or class. The library should also have a version number, which should be updated whenever a new version of the library is released.

from setuptools import setup, find_packages
 
setup(
    name="shared_library",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "pydantic>=1.8.2",
        "bcrypt>=3.2.0",
    ],
)

The package can be installed using pip and can be imported into the different microservices. The different microservices can then use the shared library to define their data schema and perform operations on data.

from shared_library import User, hash_password, verify_password
 
user = User(id="1", username="alice", password="password")
hashed_password = hash_password(user.password)
is_valid_password = verify_password("password", hashed_password)

Versioning the Shared Library

It is important to version the shared library to enable the different microservices to use the same version of the library. A version bump in the shared library should trigger a downstream dependency update in the different microservices. This can be achieved using GitHub Actions.

The GitHub Actions workflow can be defined in a YAML file located in the .github/workflows directory of the shared library repository. The workflow should be triggered whenever a new tag is pushed to the repository. The workflow should then update the version number of the shared library, create a new release, and push the changes to the repository.

name: Bump version and create release
 
on:
  push:
    tags:
      - "*"
 
jobs:
  bump-version:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
        with:
          ref: ${{ github.ref }}
      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: "3.9"
      - name: Install dependencies
        run: |
          pip install --upgrade pip
          pip install setuptools wheel twine
      - name: Bump version and create release
        run: |
          bumpversion patch --commit --tag
          git push --follow-tags
          python setup.py sdist bdist_wheel
          twine upload dist/*
          env:
          TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
          TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}

The bumpversion command is used to bump the version number of the shared library. The patch argument specifies that the patch version should be bumped. The --commit argument specifies that the version bump should be committed to the repository. The --tag argument specifies that a new tag should be created for the version bump.

The twine command is used to upload the new release to the Python Package Index (PyPI). The dist directory contains the source distribution (sdist) and binary distribution (bdist_wheel) of the shared library. The TWINE_USERNAME and TWINE_PASSWORD environment variables are used to authenticate with PyPI.

Using Dependabot for Dependency Updates

To make it easy to update the dependent repositories whenever the shared library is bumped, we can use Dependabot. Dependabot is a tool that automatically updates dependencies in your repository. It monitors your dependencies for any available updates and creates pull requests with the updated versions.

To use Dependabot, we need to create a configuration file in the .github/dependabot.yml directory of the dependent repositories. The configuration file specifies the dependencies that should be monitored for updates and how the updates should be applied.

version: 2
updates:
  - package-ecosystem: "pip"
    directory: "/"
    schedule:
      interval: "daily"
    allow:
      - dependency-name: "shared_library"
    open-pull-requests-limit: 3

The configuration file specifies that the shared library should be monitored for updates. The schedule section specifies that updates should be checked daily. The allow section specifies that only the shared library should be updated. The open-pull-requests-limit section specifies that up to 3 pull requests can be opened at a time.

Whenever a new version of the shared library is released, Dependabot will create a pull request in the dependent repositories with the updated version. The pull request can be reviewed and merged, making it easy to keep the dependent repositories up to date with the shared library. This helps reduce the likelihood of compatibility issues and ensuring that the latest features and bug fixes are used.

Conclusion

In this blog, I have described how to develop a shared library for sharing common data schemas and operations on data for a microservice architecture. The shared library was developed in Python and versioned using GitHub Actions. The shared library enables the different microservices to use the same data schema and data operations, promoting code reuse and consistency.

© Collin Mutembei.RSS