Big Data Engineer (Spark, Kafka, Python)

Big Data Engineer (Spark, Kafka, Python)

Software Guidance & Assistance





Big Data Engineer (Spark, Kafka, Python)

Details of the offer

Software Guidance & Assistance, Inc., (SGA), is searching for a
Big Data Engineer (Spark, Kafka, Python)
for a
assignment with one of our premier financial services clients in
Irving, TX


Experience with streaming in general (e.g. spark streaming) and Kafka/Confluent in particular

Experience with Data Streaming Standards and Design Patterns as well as use-cases

Development and support of Kafka integrations, including topics, producers, consumers and streaming (KStream and KTable) applications

Development and support of Confluent connect, confluent KSQL, and confluent Schema registry

Building data pipelines using Kafka

Distributed computing experience using Python and Py-Spark

Good understanding of Cloudera Big Data technologies and HDFS file formats like Avro and Parquet

Working knowledge of Java / Scala

Work with the Business Analysts and Customers throughout the requirements process to properly understand the long term goals of the program

Communication of new technologies, best practices, etc. to testers, and managers

Support continuous improvement by investigating alternatives and technologies and presenting these for architectural review

Experience with Agile methodologies, development approaches, and tools (e. g. JIRA)

Mentoring and peer review of designs and coded implementations

Expected to participate in several implementation phases of product development cycle design, scoping, planning, developing, implementation and test

Ensure the performance quality and responsiveness of applications

Investigate issues reported by testing teams to determine impact, root cause, and solve them

Identify and correct bottlenecks and fix bugs to optimize performance

Work in a fast-paced environment that fosters growth and learning

Verbal and written communication is clear and concise

Clearly articulate development and timeline dependencies

Improve team development related processes that accelerate delivery, drive innovation, lower costs, and improve quality

Ensure self and peers are actively seeking ways to objectively measure productivity. This will include a focus on automating code quality, code performance, unit testing, and build processing in the CI/CD pipeline (RTC, Jenkins, RLM)



BS/BA degree or equivalent combination of education/experience

Senior level experience in an Apps Development role. Demonstrated strong execution capabilities

5+ years' experience in Python development.

5+ years of experience data integration

3+ years of hands on Kafka/Confluent/Data Streaming development/operational platform experience

Experience in working in with Kafka connectors, Producer and consumer APIs

Strong hands on expertise in debugging and fixing streaming jobs by looking debug logs and configuration tuning

Strong understanding of Kafka architecture including offset management, partition strategy

Good understanding of Kafka Security with Kerberos

Good understanding of Streaming message delivery semantics

Good understanding of Spark framework

Strong understanding of streaming message formats like Avro and streaming semantics

Expert knowledge of browser behavior and compatibility, cross-browser issues and solutions, and coding for search engine optimization

Experience with distributed source control systems, preferably BitBucket

Experience developing high performance server side components



Master's Degree in Computer Science or equivalent field

Strong business-facing skills

Cloudera Hadoop experience

Prior financial services or banking experience

Previous experience with Confluent Kafka is a plus

SGA is a Certified Women's Business Enterprise (WBE) celebrating over thirty years of service to our national client base for both permanent placement and consulting opportunities. For consulting positions, we offer a variety of benefit options including but not limited to health & dental insurance, paid vacation, timely payment via direct deposit. SGA accepts transfers of H1 sponsorship for most contracting roles. We are unable to sponsor for Right-to-Hire, Fulltime, or Government roles. All parties authorized to work in the US are encouraged to apply for all roles. Only those authorized to work for government entities will be considered for government roles. Please inquire about our referral program if you would like to submit a candidate for any of our open or future job opportunities. SGA is an EEO employer. We encourage Veterans to apply. To view all of our available job postings and/or to learn more about SGA please visit us online at



  • Java
  • Python

Devops cloud engineer (infrastructure)

Trigyn's Direct Government Client has an immediate need for an DevOps Cloud Engineer in Baltimore, MD The Infrastucture Developer is responsible for treating...

From Trigyn Technologies, Inc. - Maryland

Published a month ago

Microsoft dynamics developer

Microsoft Dynamics Developer This position includes support of all aspects of business lifecycle, client lifecycle, and deal management process including CRM...

From Stratus Technology Services, Llc - New York

Published a month ago

Associate business intelligence developer

Title: Associate Business Intelligence Developer Location: Springfield, OR Type: Perm Job #: DE36074167 Note: This job is not open to C2C or 3rd party...

From Ignw - Oregon

Published a month ago

Crm tester

Trigyn's direct government client has an immediate need for CRM Tester in Manhattan, NY. Description: Siebel Application Enhancements is in progress to convert...

From Trigyn Technologies, Inc. - New York

Published a month ago