Infrastructure Costs Optimization
We singled out unused or ineffective elements of the system and got rid of them, redistributed computing power, and received various discounts from infrastructure providers. This allowed us to decrease the infrastructure costs from ~$40k to $22k.
IaC Implementation with AWS CloudFormation
We described, implemented, and documented new services: VPC, S3, CloudFront, ECS, LB, ACM, RDS, ECR, WAF, IAM, OpenSearch, Сloudwatch, Cognito. The resulting infrastructure is codified, transparent, documented, and suitable for automation and reuse.
Infrastructure Migration between Data Centers
We developed an infrastructure migration strategy taking into account the requirement to limit downtime to 3–4 hours. We migrated the infrastructure that was built on the Proxmox VE virtualization system (10 VMs, 10 Tb in total, including a 2 Tb 1С database) from Heztner data center to netrack. Transferred sensitive data from a 5 Tb Heztner storage to a separate server and set up backups for databases and sensitive data. The system downtime due to migration was less than 2 hours.
Logs Delivery
We built a system for guaranteed log delivery by implementing Kafka into the log collection system; we clustered and optimized ELK (standard load ~10k logs per second). As a result, we solved the problem of logs being lost due to recurrent network issues caused by high loads.
Fault Tolerance Improvement
By clustering critical system components, we increased the infrastructure’s stability and allowed its parts to be maintained separately without disrupting its work or stopping business processes.
Latency Reduction and Performance Increase
We reduced latency by fine-tuning Linux, implementing developers' ideas and new technologies. As a result, we were able to increase trading volumes and improve efficiency.
PCI DSS Compliance
We filled out a questionnaire (the infrastructural part); divided environments by sub-accounts in AWS organization; created and implemented a role model/matrix for the project (IAM), implemented MFA, granular access permissions for user groups, and a role model for the database; implemented a Password Policy, AWS WAF, secrets management (environment variables, access credentials), performed key rotation and implemented AWS KMS; configured SG. We updated and refactored VPN channels for contractors, implemented integration with processing services through encrypted tunnels. The changes made in the infrastructure improved the system’s security and allowed it to successfully pass the PCI DSS audit.
Fast Infrastructure Deployment
We significantly reduced the time needed for infrastructure deployment by implementing the IaC approach: wait time for new instances was reduced from 12–24 hours to 15 minutes. We also removed infrastructural blockers for hypotheses testing, which enabled the system's horizontal and vertical scaling in just tens of minutes.
Time to Market Decrease
We optimized the assembly and deployment of the trading bot from 4 to 1.5 minutes that increased the accuracy of responds to market changes and reduced the time needed for debugging and development.
Load Testing
We analyzed the system performance under load, checked its compliance (and non-compliance) with requirements and provided recommendations for optimization. We collected data on system performance and response time and provided recommendations for protection from DDoS attacks. By implementing our recommendations, the customer was able to ensure stable operation of the system during peak load periods.