Software Engineer (Data Platforms)

Technology - TB


True Beacon, backed by the Zerodha promoters, is a technology-driven investment firm. We allocate client capital in public markets and provide a platform for top-tier talent to create novel investment products across various asset classes. The engineering team is responsible for building systems that drive investment decision-making and play a crucial role in various business functions. Engineers on the team utilise a mix of in-house and open-source software to design, develop, test, and deploy solutions across quantitative research, trading, and firm operations. While prior experience with financial data is beneficial, no finance background is required for this role. 


Contributions to open-source projects and personal projects are highly valued.


Responsibilities


  1. Design, develop, and deploy data models and frameworks for processing financial data across vendors, markets, and historical timeframes at a global scale, which can be used with alternative datasets
  2. Use open-source tools for data quality, cleaning, ETL processes, and data lake management to support investment research, analytics, and client reporting
  3. Contribute to the development across the stack, from server-side to front end to create simple solutions that solve business problems
  4. Drive process improvements by automating tasks and ensure adherence to development best practices, maintaining high code quality and data integrity standards.


Skills

  1. Prior experience in building scalable, performant and reliable data systems
  2. Strong working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
  3. Proficient in at least one of Java, Python or Scala 
  4. Experienced in engineering pipelines using big data technologies (Hive, Presto, Spark, Flink)
  5. Experience with AWS cloud services for data engineering and analytics tools.
  6. Basic knowledge of full-stack development, including any server-side language and frontend stack.
  7. Competency in using git for version control 
  8. Experience deploying and maintaining open source software in a Linux Environment.


Good to Have


  1. Foundational understanding of financial data processing, real-time and historical data analysis.
  2. Experience with open-source data collection tools (e.g., Apache NiFi, Airbyte) for effective data ingestion.
  3. Knowledge of open-source data warehousing tools (e.g., ClickHouse, Druid, Pinot, Spark) for financial data management.
  4. Familiarity with transformation and orchestration tools (e.g., dbt, Prefect, Airflow, Apache Beam) for data processing and workflow automation.
  5. Experience with BI tools (e.g., Metabase, Apache Superset) for insightful data visualisation and reporting.
  6. Understanding of data quality tools (e.g., Great Expectations) for ensuring data reliability.
  7. Experience with any of these tools across various data engineering steps is seen as beneficial, though not all are required.


Qualifications


  1. A proven track record or a strong interest in Data Engineering, especially within financial services or a related sector, can be considered.
  2. Demonstrable expertise in building and optimising data pipelines and architectures, particularly on AWS.
  3. Formal education in Computer Science, Engineering, or related fields is not mandatory; practical experience and a portfolio of relevant projects are highly valued.


What We Offer

  1. High ownership role at an investment firm where technical leaders drive investment strategies and business decisions.
  2. Opportunity to work with state-of-the-art technologies and contribute to impactful projects in the financial sector.
  3. Competitive compensation package, with a focus on fostering learning, professional growth and career development.