Hi
Hope you doing well.
Please review the job description and let me know if you are interested or available for this position and do share your updated resume to pratima.kumari@testingxperts.com
Job role : Lead Databricks Engineer
Location: Iselin NJ / New York, NY Hyrbid (2-3 Days)
Job Description:
Must have : Expert in Databricks , Pyspark , python
Strong knowledge on data bricks architecture and tools
Have experience of task and wf jobs creations in data bricks.
Deep understanding of distributed computing and how to use spark for data processing.
SQL and pyspark : strong command over querying databases and proficiency in pyspark.
Cloud platform: Preferred Azure for data bricks deployment.
Qualifications
Bachelor's degree in Computer Science, Engineering, or related field, or equivalent work experience
5+ years of experience in data engineering with Databricks and Spark
Proficient in SQL and Python and Pyspark
Experience with Azure Databricks Medallion Architecture with DLT, Iceberg
Financial/Corporate Banking context would be a plus
Experience with data integration and ETL tools, such as Azure Data Factory
Experience with Azure cloud platform and services
Experience with data warehouse and data lake concepts and architectures
Good to have experience with big data technologies, such as Kafka, Hadoop, Hive, etc
Strong analytical and problem-solving skills
Excellent communication and teamwork skills
Responsibilities
Design, develop, and maintain data pipelines using Databricks and Spark, and other cloud technologies as needed
Optimize data pipelines for performance, scalability, and reliability
Ensure data quality and integrity throughout the data lifecycle
Collaborate with data scientists, analysts, and other stakeholders to understand and meet their data needs
Troubleshoot and resolve data-related issues, and provide root cause analysis and recommendations
Document data pipeline specifications, requirements, and enhancements, and communicate them effectively to the team and management
Create new data validation methods and data analysis tools, and share best practices and learnings with the data engineering community
Implement ETL processes and data warehouse solutions, and ensure compliance with data governance and security policies
Kind Regards,
Pratima Kumari (Executive Recruiter)
Corporate Office: 650 Wilson Lane, Suite 201, Mechanicsburg, PA 17055
E: pratima.kumari@testingxperts.com
Skype: pratima.kumari.tx@outlook.com
W:
Please consider the planet before printing | Switch off as you go |Recycle always | Your child deserves a better world
CONFIDENTIALITY NOTICE : This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original messages. Whilst we run the anti-virus software, you are solely responsible for ensuring that any email or attachment you receive is virus-free. We disclaim liability for any damage you suffer as a consequence of receiving any such virus.
...-Term Disability, Long-Term Disability, EAP, Tuition Reimbursement & Awesome Travel Perks! Job Overview: To maintain and repair painted; wall covered, stained and varnished surfaces of building, furniture and equipment throughout the facility in order to project a neat...
...Job Title:Incident Response Analyst (Cyber Security Analyst) Hospital Location: Akron, OH Shift Details : 08:00 - 17:30 Pay Rate : ($70/hr.)(All Inclusive) Contract to Hire opportunity Summary: Must have healthcare experience - coming...
...Speech-language Pathologist Assistant ( SLP A) - Outpatient Clinic We have an immediate need for a Speech-language Pathologist Assistant (SLPA) who is seeking a permanent position in an Outpatient setting. Job Description As a Speech-language Pathologist Assistant...
...2024 11:59 PM Eastern Bargaining Unit UNITED AUTO WORKERS (UAW)... ...recognized resource evaluation consultant for the healthcare programs implemented by the Cancer Prevention... ...and Information Position Location/Remote Office: The physical location of this...
This is a remote position. We are looking for energetic individuals who have an interest in providing our Client\Customers with excellent customer service to work from home with the ****@*****.*** program. ****@*****.*** is only available in AL, FL, GA, MD, MS, NC, ME, PA...