Valitive
Valitive
B2B consumer data handling application is designed over microservice architecture for processing, cleansing, filtering and transferring data related to the entire population of Sweden.
Size of the project
Technologies / Tools
CI/CD Solution for Microservices
All the application repositories are managed on GitLab along with a clean Git Flow pattern. Teamcity & AWS CodePipeline is used for CI/CD automation. CI/CD Pipeline is designed to build, check, and run automated tests of spring boot apps. Once the build is successful, it is packaged as a Docker image and registered in ECR for releasing a new version. All the new containers are deployed to ECS seamlessly with zero downtime of the live system.
Automated Data Cleansing Process
This automation aims to get updated data from different sources and make those available in a structured B2B exchangeable format that involves various data merging & verification processes.
Beyond the DevOps Role
Our DevOps team worked beyond their duties and designed a fully automated data cleansing process that connects around 5+ data sources with the volume of around 120M data using Apache Kafka. It uses different Kafka brokers for different data types and produces data to Mysql, Elastic Index & static data files. This Kafka setup runs on a Kubernetes cluster and processes data in a per-scheduler.