Description

We are looking for a Mid-Level Python Backend & Data Processing Engineer to build and maintain backend services and high-volume data processing pipelines. The primary data sources are large plain-text log files, requiring efficient parsing, validation, transformation, storage, and reporting.

The role involves working across backend APIs, data pipelines, background processing, reporting, and dashboard integration. The ideal candidate should be technically strong, proactive, ownership-driven, and comfortable communicating directly with clients and cross-functional teams

Job Responsibilities

• Develop scalable Python applications and REST APIs using Django/DRF or FastAPI.
• Design and maintain data ingestion pipelines for large plain-text log files.
• Develop efficient streaming/batch processing to handle large datasets without excessive memory usage.
• Analyze log formats and implement robust parsing, validation, and transformation logic.
• Handle malformed, duplicate, and invalid records with appropriate error tracking.
• Design reliable, restartable, and idempotent data-processing workflows.
• Work with PostgreSQL and analytical/data-processing technologies as required.
• Develop background jobs using Celery, Redis, or similar technologies for data processing, notifications, and report generation.
• Generate and maintain reports in formats such as CSV, Excel, and PDF.
• Develop APIs to support web dashboards, reporting, filtering, search, and data visualization.
• Optimize database queries, data-processing workflows, and overall application performance.
• Write unit and integration tests and maintain clean, maintainable code.
• Implement logging, monitoring, error handling, and basic production troubleshooting.
• Collaborate with frontend and other engineering teams to deliver end-to-end features.
• Proactively identify technical risks, edge cases, performance bottlenecks, and opportunities for improvement.
• Take ownership of assigned features and workflows from requirement understanding through development, testing, deployment, and support.
   

Required Technical Skills

• 3–5 years of professional Python experience.
• Strong understanding of Python, OOP, type hints, exception handling, generators, and file processing.
• Experience with Django/DRF, FastAPI, or similar backend frameworks.
• Strong SQL and PostgreSQL experience.
• Experience with Pandas and data processing.
• Experience parsing and processing CSV, JSON, plain-text, or log data.
• Understanding of ETL/ELT, data validation, batch processing, and data pipelines.
• Experience with Celery, Redis, or similar background-job systems.
• Good understanding of REST APIs, authentication, and API design.
• Experience with Git, Pytest, Docker, and CI/CD practices.
• Understanding of database indexing, transactions, query optimization, and performance considerations.
   

Communication & Client Interaction

• Strong verbal and written communication skills, with the ability to clearly explain technical concepts to both technical and non-technical stakeholders.
• Comfortable with direct client communication, requirement clarification, technical discussions, and progress updates.
   

Ownership & Proactiveness

• Take end-to-end ownership of assigned features and responsibilities and remain accountable for delivery and outcomes.
• Proactively identify issues, risks, blockers, and opportunities for improvement rather than waiting for instructions.
   

Problem Solving & Analytical Thinking

• Strong analytical and problem-solving mindset, with the ability to independently investigate issues and identify root causes.
• Able to break down complex problems, evaluate alternatives, and propose practical and effective solutions.
   

Engineering Excellence

• Write clean, readable, maintainable, and testable code while following appropriate software engineering best practices.
• Consider performance, scalability, reliability, security, and maintainability when designing and implementing solutions.
   

Continuous Learning & Growth Mindset

• Demonstrate genuine curiosity and a passion for continuous learning, keeping up with new technologies, tools, and engineering practices.
• Ability and willingness to quickly learn and apply new technologies or approaches when project requirements demand them.
   

Collaboration & Team Contribution

• Collaborate effectively with developers, QA, DevOps, PMs, clients, and other stakeholders to deliver high-quality solutions.
• Share knowledge, provide constructive feedback, and contribute to improving team practices and engineering standards.
   

Nice to Have

• Experience with high-volume or large-scale log/event data.
• Experience with ClickHouse, DuckDB, Polars, PyArrow, or Apache Spark.
• Knowledge of Parquet and columnar data storage.
• Experience with Kafka, RabbitMQ, or similar messaging systems.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Basic knowledge of React/TypeScript and frontend-backend integration.
• Experience working directly with international clients or distributed teams.
• Experience mentoring junior developers or taking technical ownership of a component or service.
   

What We Expect

The ideal candidate can take a data-processing requirement from raw log ingestion → validation → processing → storage → reporting/API → dashboard, while considering reliability, performance, scalability, and maintainability.
 

Beyond technical skills, we expect the candidate to:

• Take ownership rather than simply complete assigned tasks.
• Communicate clearly and confidently with clients and team members.
• Be proactive in identifying and solving problems.
• Demonstrate strong attention to data quality and engineering standards.
• Be willing to learn, adapt, and continuously improve.
• Understand the business objective behind technical requirements and make practical engineering decisions.
• Contribute positively to the team through collaboration, knowledge sharing, and constructive feedback.

• Attractive Salary for deserving candidates
• Medical Coverage
• Yearly Salary Review
• Weekly two holidays
• Provident fund
• Gratuity fund
• Two festival bonus
• Monthly Performance Bonus
• WPPF.