Required Technical Skills Python Development: 3-5 years of professional Python development including: - Advanced proficiency in Python 3.x - Experience with data processing libraries (Pandas, Num Py, Polars) - Knowledge of Python frameworks for APIs (Fast API, Flask) - Experience with async/await patterns and concurrent programming - Understanding of Python best practices, type hints, and documentation standards - Experience with Python package management (pip, poetry, conda)Azure CI/CD: Strong experience with: - Azure Dev Ops Pipelines (YAML and Classic) - Git workflows and branching strategies (Git Flow, trunk-based development) - Automated testing integration (pytest, unittest) - Artifact management and versioning - Pipeline templates and reusable pipeline components - Service connections and secure credential management Database Skills: Expert knowledge of: - SQL query optimization and performance tuning - Database indexing strategies - Transaction management and concurrency control - Database migrations and schema versioning tools (Alembic, Flyway) - Connection pooling and database driver configuration - Database monitoring and diagnostics Data Transformation Tools: Proficiency with: - Pandas for data manipulation and analysis - SQL for complex joins, aggregations, and window functions - Azure Data Factory data flows and mapping - Data validation libraries (Great Expectations, Pandera) - JSON and XML parsing and transformation - Regular expressions for pattern matching and extraction Development Tools: Working knowledge of: - Python IDEs (VS Code, Py Charm, Jupyter) - Git version control and pull request workflows - Debugging tools and profiling tools - Linux command line and bash scripting - Docker for containerization - Azure CLI and Power Shell for automation Required Soft Skills Strong analytical and problem-solving abilities Excellent communication skills for collaborating with data analysts and business stakeholders Ability to translate business requirements into technical data solutions Self-motivated and able to work independently on complex data challenges Strong attention to detail for data quality and accuracy Ability to document data pipelines, schemas, and transformation logic Willingness to learn new data technologies and adapt to evolving requirements Ability to work as part of a distributed team and collaborate with stakeholders at all levels Preferred Qualifications Certifications: - Microsoft Certified: Azure Data Engineer Associate (DP Microsoft Certified: Azure Developer Associate (AZ Microsoft Certified: Azure Database Administrator Associate (DP Microsoft Certified: Azure Fundamentals (AZ-900)Additional Experience: - Experience with Apache Spark for big data processing - Knowledge of data visualization tools (Power BI, Grafana, Plotly) - Experience with machine learning pipelines and MLOps - Understanding of data governance and data cataloging tools - Experience with Azure Purview for data discovery and classification - Previous work with Io T data platforms and real-time analytics - Experience in railway, transportation, or critical infrastructure industries - Knowledge of Agile/Scrum development methodologies Specific Responsibilities for Data Application Support Maintain and enhance Python-based data applications running in Azure environments Design and implement ETL/ELT pipelines for ingesting data from multiple sources Develop and optimize SQL queries and stored procedures for data processing Build and maintain CI/CD pipelines for automated deployment of data applications Implement data transformation logic using Pandas, SQL, and Azure Data Factory Ensure data quality through validation, cleansing, and error handling mechanisms Monitor data pipeline performance and troubleshoot failures Optimize database queries and indexes for improved performance Collaborate with Dev Ops engineers on infrastructure and deployment automation Document data schemas, transformation logic, and pipeline workflows Participate in code reviews and promote best practices for data engineering Support integration of Io T data streams into data warehouses and analytics platforms Implement data security measures including encryption, access control, and auditing Create and maintain automated tests for data pipelines and transformations Provide technical guidance on data architecture and design patterns