Technologies we work with

01 Stack

Big data and distributed processing

• Apache Spark and PySpark
• Hadoop, HDFS, YARN and MapReduce
• Apache Hive and HBase
• Apache Kafka and Kafka Connect
• Apache Flink and Spark Structured Streaming
• Apache NiFi and Apache Airflow
• Apache Iceberg, Delta Lake and Apache Hudi
• Trino and Presto
• Apache Druid and Apache Pinot
• Apache Ranger and Apache ZooKeeper
• Cloudera ecosystem technologies
• Snowflake
• Databricks

Older Hadoop components such as Pig, Oozie and Ambari still matter where an environment needs maintenance, migration or interoperability.

02 Stack

Cloud platforms

• Amazon Web Services : EC2, S3, RDS, Aurora, DynamoDB, EMR, Glue, Redshift, Athena, Lambda, ECS, EKS, VPC, Kinesis, Amazon MSK, SageMaker, Bedrock, Lake Formation, IAM, KMS, CloudTrail and CloudWatch
• Microsoft Azure : Azure Data Factory, Azure Databricks, Synapse Analytics, Azure Data Lake Storage, Event Hubs, Functions, AKS, Azure Machine Learning, Microsoft Foundry and applicable Azure AI services, Microsoft Fabric, Power BI, Entra ID, Key Vault, Azure Monitor and Defender for Cloud
• Google Cloud : BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Cloud Composer, Dataplex, Vertex AI, Cloud Run, Google Kubernetes Engine, Cloud SQL, Spanner, Cloud Logging, Cloud Monitoring, IAM and Secret Manager

Cloud architecture balances managed-service benefits, portability, operational simplicity, security and commercial constraints.

03 Stack

Databases and data storage

• Relational : PostgreSQL, MySQL, Microsoft SQL Server, Oracle Database, Amazon Aurora and Cloud SQL
• NoSQL and distributed : MongoDB, Apache Cassandra, DynamoDB, Azure Cosmos DB and Redis
• Engineering practice : Advanced SQL, PL/SQL, T-SQL, schema design, indexing, partitioning, query optimization, database migration, change data capture, replication and transaction management

04 Stack

AI, machine learning and computer vision

• PyTorch, TensorFlow, Keras and JAX
• scikit-learn, XGBoost and LightGBM
• Hugging Face Transformers, spaCy and NLTK
• OpenCV, Pillow and torchvision
• MLflow and Kubeflow

05 Stack

Generative AI and retrieval

• LangChain and LangGraph
• LlamaIndex and CrewAI
• Foundation model APIs
• Amazon Bedrock, Google Vertex AI and applicable Microsoft AI services
• PostgreSQL with pgvector
• OpenSearch and Elasticsearch
• Qdrant, Weaviate and Milvus

Around them: semantic retrieval, hybrid search, reranking, document parsing, embedding generation, prompt versioning, structured outputs, model routing and evaluation.

06 Stack

Analytics and visualization

• Power BI, Tableau and Looker
• Amazon QuickSight, Apache Superset and Grafana
• SSAS, SSMS, DAX and Power Query
• Semantic modeling, KPI governance and automated reporting

07 Stack

DevOps and infrastructure automation

• Docker
• Kubernetes and Helm
• Terraform, OpenTofu and Terragrunt
• Ansible
• GitHub Actions, GitLab CI/CD and Jenkins
• Argo CD
• Prometheus, Grafana and OpenTelemetry
• Policy-as-code and infrastructure testing

08 Stack

Programming and application engineering

• Python
• SQL, Scala and Java
• JavaScript, C, C++ and C#
• React and TypeScript
• FastAPI, Flask, Node.js, Spring Boot and .NET

Practices: REST and event-driven APIs, automated testing, code review, modular architecture, dependency management, secure coding and maintainable design.

09 How we choose

How do we choose technology?

• Business requirements and intended outcomes
• Workload size, latency, throughput and availability
• Security, privacy and regulatory constraints
• Compatibility with existing systems
• Team expertise and operational capacity
• Licensing and total cost of ownership
• Maintainability and lifecycle requirements
• Vendor dependence and portability
• Reliability and supportability

We aim for a coherent environment that fits the purpose, not the largest possible stack.