DevOps Engineer - IBM/USAA
Location: San Antonio, TX
Tools & Technologies:
• Apache Kafka (Self-managed or MSK)
• AWS Managed Apache Flink
• Amazon EC2, S3, RDS, and VPC
• Terraform/CloudFormation
• Docker, Kubernetes (EKS)
• ELK Stack, CloudWatch
• Python, Bash
Skills & Expertise:
1. AWS Managed Services:
◦ Proficiency in AWS services like Amazon MSK, Kinesis, Lambda, S3, EC2, RDS, VPC, and IAM.
◦ Experience with infrastructure management using Terraform or AWS CloudFormation.
2. Apache Flink:
◦ Understanding of real-time stream and batch data processing with Apache Flink.
◦ Familiarity with Flink's integration with Kafka and managing Flink clusters on AWS.
3. Kafka Broker (Apache Kafka):
◦ Strong knowledge of Kafka architecture, brokers, topics, partitions, producers, and consumers.
◦ Experience with Kafka management, monitoring, scaling, and optimization, particularly with Amazon MSK or self-managed Kafka clusters.
4. DevOps & Automation:
◦ Expertise in automating deployments and infrastructure provisioning.
◦ Familiarity with CI/CD pipelines using tools like Jenkins, GitLab, GitHub Actions, CircleCI.
◦ Experience with Docker and Kubernetes for containerizing and orchestrating applications in the cloud.
5. Programming & Scripting:
◦ Strong scripting skills in Python, Bash, or Go for automation tasks and data pipeline integrations.
6. Monitoring & Performance Tuning:
◦ Proficiency with monitoring tools like CloudWatch, Prometheus, Grafana.
◦ Expertise in optimizing data pipelines for scalability, fault tolerance, and performance.
Responsibilities:
1. Infrastructure Design & Implementation:
◦ Design and deploy scalable, fault-tolerant real-time data processing pipelines using Apache Flink and Kafka on AWS.
2. Platform Management:
◦ Manage and optimize Kafka clusters (MSK or self-managed) and Flink jobs on AWS infrastructure for low-latency, high-throughput processing.
3. Automation & CI/CD:
◦ Automate infrastructure provisioning and deployment using Terraform, CloudFormation, or other tools.
◦ Integrate new applications into CI/CD pipelines for real-time data processing.
4. Collaboration with Data Engineering Teams:
◦ Work with Data Engineers, Data Scientists, and DevOps teams to ensure smooth integration and high performance of real-time data systems.
5. Security and Compliance:
◦ Implement security for Kafka and Flink clusters, ensuring compliance with regulatory standards (e.g., GDPR, HIPAA).
6. Optimization & Troubleshooting:
◦ Optimize Kafka and Flink performance, troubleshoot issues related to message delivery, job failures, or AWS service outages.
...BC Support Consultant a0M1i00000JVvd3.1_1742236128 Job Title: Business Central Support Consultant Location: Remote / Hybrid / On-Site (Based on Company Needs) Job Type: Full-Time / Contract : We are seeking a skilled Business Central Support Consultant...
...Float Support Specialist | Intensive Behavioral Services Locations: Burnsville | Apple Valley | Brooklyn Park Compensation: $25-$30/hour Professional Opportunity in Behavioral Support Services About BrightPath: We are a dedicated organization providing comprehensive...
...manufacturing, advanced fabrication and building services. Led by Rick and Jon Kinning, RK... ...establishing scope of work with Account Managers or Inside Sales when required.~Complete... ...industrial plumbing, mechanical, process and HVAC contracting.RK Steel: custom...
...ZULU TRANSLATOR Job Posting - Prisma International We are onboarding experienced Zulutranslators to join our independent contractor team.You will be providing written translation from English into Zulu. To be considered for hire, you must meet all minimum requirements...
...GLOBAL PACIFIC SUPPORT is looking for an experienced and dedicated Go High Level CRM Onboarding Specialist to join our team. In this... ...plus. Self-motivated and able to work independently in a remote environment. Bachelors degree in Business, Marketing, or...