Lead Data Engineer

Job Description:

Lead Data Engineer

Description

Job Description

Office of Information Technology

Annual Salary Range Lead IT Analyst: $112,497 - $149,972

FLSA Exempt / Union Represented

This position allows for up to four days of remote work per week

About the Office

The Office of Information Technology (IT) is responsible for enabling State Bar's internal and external stakeholders by the management, implementation, and maintenance of an organization's technology to support of State Bar's mission and goals. The office's primary goals are to build and maintain functional capabilities, to support innovation and to ensure that all systems are running smoothly, efficiently, and securely to support the needs of the organization and its stakeholders.

Job Overview

The Office of Information Technology is seeking a Lead Data Engineer to join our dynamic team at the State Bar of California. In this role, you will define and drive the modernization of our data infrastructure using Microsoft cloud technologies. As the technical authority for the Bar's data platform, you will lead the development of scalable, automated data pipelines, establish governance standards, and guide cross-functional teams to deliver data driven solution that support Analytics, AI, compliance, and operational efficiency.

The Ideal Candidate

The ideal candidate is a hands-on technical leader with experience in building scalable, cloud-based data solutions. This individual brings a strong understanding of modern data architectures, data integration patterns, and secure design principles and is capable of driving architectural decisions that align with long-term business goals.

They have demonstrated success in leading cross-functional collaboration, setting standards for data quality and governance, and building reusable data frameworks that support analytics, automation, and decision making.

A clear understanding of compliance standards (e.g.,PII, PCI, HIPAA, CPRA) and experience applying those principles in large scale environments is essential. Microsoft certification in one or more of the following is strongly desired: Azure Data Engineer Associate; Power BI Data Analyst Associate; Azure Solutions Architect Expert; or any other data analysis-related certifications.

Definition

With minimum direction, the Lead IT Analyst provides technical leadership and direction for major IT projects, and/or serves as the technical authority for one or more related functions, including: network operating systems; software applications; data analytics applications, project/program management; product management; desktop support; system engineering; network infrastructure; database management; cloud services administration; and web server management. The Lead IT Analyst performs advanced, highly specialized IT work involving the business requirements, analysis, product roadmaps, planning, project management, architecture, designing, implementation, maintenance, troubleshooting, and/or enhancement of complex IT systems and procedures, and may serve in a lead capacity over other IT analysts and support staff.

Distinguishing Characteristics

The Lead IT Analyst is the highest level in the IT Analyst series and may be assigned to a function as an individual technical expert, project team leader. When assigned as a technical expert, performs work requiring a very high level of technical knowledge of a specific area, or ability to integrate at a high level the knowledge of several areas, while working with a significant amount of independent authority and judgment. When assigned as a project team leader, manages and provides technical leadership of projects involving large-scale, complex, and highly analytical tasks. When assigned as a supervisor, may perform work comparable to a project team leader, but with direct supervision over subordinate IT analysts and support staff.

The Lead IT Analyst is distinguished from the Senior IT Analyst in that the former performs the most complex IT operations and applications-related duties, provides technical leadership and direction to lower-level IT staff, and assumes responsibility for completion of major IT projects in accordance with the State Bar's needs. The Lead IT Analyst also provides strategic input and guidance within the incumbent's area of specialization.

Role-Specific of Essential Duties

Duties may include, but are not limited to the following:

• Lead the design, development, and maintenance of scalable data pipelines using Azure Data Factory, Synapse Analytics, and Data Lake Storage to ingest, transform, and publish structured and unstructured data.
• Drive the automation of end-to-end and event-driven workflows using Azure Logic Apps, ensuring real-time or scheduled data synchronization and integration with external systems.
• Direct enterprise-level data modeling and transformation efforts to support reporting, dashboards, and analytics, including validation and quality checks to ensure consistency and reliability.
• Guide and collaborate with IT and business teams to gather data requirements, define transformation logic, and develop shared documentation such as the data dictionaries.
• Develop and oversee enhancements of the Azure-based data architecture, focusing on scalability, reusability, and long-term maintainability.
• Establish and implement data governance standards, including metadata management, lineage tracking, and data privacy and security controls aligned with organizational compliance needs.
• Optimize datasets for Power BI and promote self-service analytics through clean, reusable data models and standardized practices.
• Manage, monitor, troubleshoot, and tune pipeline performance, proactively resolving issues and establishing observability standards for operational reliability.
• Ensure comprehensive documentation and lead knowledge transfer efforts to support long-term IT business and user sustainability and system adoption.
• Maintain and support databases for custom and legacy applications, including performance tuning, backup and recovery, indexing, and monitoring.
• Spearhead the optimization of stored procedures, ETL jobs, and scheduled tasks, ensuring efficiency and reliability across both modern and legacy environments.
• Partner with application teams to troubleshoot and resolve data-related issues, implement fixes, and ensure data integrity in legacy platforms.
• Lead enterprise data migration efforts aligning strategies for access control, security, and compliance as data moves from legacy to modern platforms.
• Build and maintain an enterprise data dictionary and catalog to improve data discoverability and consistency across the organization.
• Lead and implement data security and compliance standards, across the Bar's data footprint.

General Examples

Examples of Essential Duties

• When assigned to the IT Application Engineering, Product Management, or Program Management Teams, duties may include, but are not limited to the following:
• Analyzes and evaluates existing IT applications, products, and systems for cost effective and efficient operation, and identifies opportunities for modifications and improvements; prepares feasibility studies and project proposals accordingly.
• Plans, designs, tests, develops, and integrates solutions based on operational needs to support the business roadmap.
• Determines project schedules and allocation of resources for projects; determines operating characteristics, requirements; monitors the utilization of resources to ensure optimal effectiveness and efficiency; resolves production issues; and ensures projects, tasks and other activities are completed on schedule and in a timely and efficient manner.
• Monitors and maintains existing IT applications and products to ensure optimal functionality.
• Acts as the leader, supervisor, and primary point of contact for their assigned team.
• Coordinates closely with other IT staff and business partners to accomplish the collective goals and objectives of assigned programs, projects, and activities.
• Develops, maintains, and enforces various standards, best practices, procedures, policies and standard operating processes.
• Serves as the subject matter expert, mentor, guide in their respective area of expertise.

When assigned to the IT Operations and Infrastructure Team Duties may include, but are not limited to the following:

• Supports the IT Director and IT Management team in the development, planning, estimating, and implementation of goals, objectives, and other activities for modifications and improvements.
• Acts as the leader, supervisor, and primary point of contact for the IT Operations and Infrastructure Team.
• Oversees the desktop support team and/or infrastructure team to ensure that all known incidents and Zendesk tickets are accurately identified and assigned efficiently; ensures that any tickets that cannot be resolved within agreed timeframes are properly escalated.
• Oversees the desktop support team and/or infrastructure team, and manages the ticket queue and incoming requests related to IT issues.
• Administers and supports configurations and maintains records of end-user's equipment. Endpoints include laptops, mobile devices, printers, AV systems, and other related peripherals.
• Oversees the infrastructure configuration and management to support State Bar data retention, security, business continuity, disaster recovery planning/testing, and information risk management policies.
• Coordinates closely with IT Application Engineering, IT PMO, and IT Product Management teams to accomplish the goals and objectives of assigned programs, projects, and activities.
• Collaborates with end-users and the system engineer team to ensure optimal functionality and system effectiveness.

Role Specific Employment Standards

Knowledge of:

• Advanced functions of computer information systems, including the core hardware and software components that comprise complex IT systems and networks, as well as personal computer applications, word processing, and spreadsheet programs.
• Analysis and evaluation techniques to plan, design, integrate, implement, maintain and troubleshoot complex IT systems and networks.
• Techniques and practices of supervision, leadership, project management, and training.
• Project management and information systems development, design, and maintenance.
• Technical, operational, and programming problem-solving analysis.
• Principles of Agile application development methodologies.
• Problem identification, analysis and evaluation.
• Principles of effective writing and verbal presentation.

Desired Knowledge:

• Proficient in building scalable data pipelines using Azure Data Factory, Synapse Analytics, Azure Data Lake, and Logic Apps.
• Strong SQL skills with experience in data transformation, validation, and optimization for reporting and analytics.
• Skilled in dimensional modeling and modern data platform design to support dashboarding and analytical use cases.
• Familiar with data quality frameworks, including validation, anomaly detection, and exception handling processes.
• Experience with metadata management, data cataloging, and governance tools such as Azure Purview (or equivalents).
• Familiarity with CI/CD practices and version control systems such as Git.
• Demonstrated ability to translate business needs into technical requirements and collaborate effectively across IT and business teams.
• Strong communication, documentation, and stakeholder engagement skills to support alignment and knowledge sharing.
• Data engineering languages such as Python.
• Knowledge of data security best practices including encryption, data masking.
• Familiarity with data visualization tool Power BI.
• Technical, operational, and programming problem-solving analysis.
• Problem identification, analysis and evaluation.
• Principles of effective writing and verbal presentation.

Ability to:

• Exercise independent judgement, decisiveness, and creativity in designing, planning, troubleshooting, and integration of complex systems or networks based on the potential benefits and/or consequences.
• Analyze, evaluate, present and recommend technology methodologies and solutions.
• Solve problems with minimal disruption or impact to the organization business functions.
• Develop technology guidelines and standards.
• Develop alternative solutions within established guidelines and standards.
• Prioritize competing requests for service.
• Develop policies, procedures and standards relevant to the area of expertise.
• Interpret and implement IT policies, procedures, and standards.
• Operate modern office equipment including computer hardware, software, and internet and web-based applications.
• Compile data and prepare narrative and statistical reports.
• Communicate clearly and concisely, both orally and in writing.
• Establish and maintain effective interpersonal relationships at all organizational levels as well as with the public.
• Maintain the confidentiality of information.

General Employment Standards

Knowledge of:

• Advanced functions of computer information systems, including the core hardware and software components that comprise complex IT systems and networks, as well as personal computer applications, word processing, and spreadsheet programs.
• Analysis and evaluation techniques to plan, design, integrate, implement, maintain and troubleshoot complex IT systems and networks.
• Techniques and practices of supervision, leadership, project management, and training.
• Project management and information systems development, design, and maintenance.
• Technical, operational, and programming problem-solving analysis.
• Principles of Agile application development methodologies.
• Problem identification, analysis and evaluation.
• Principles of effective writing and verbal presentation.

Ability to:

• Exercise independent judgement, decisiveness, and creativity in designing, planning, troubleshooting, and integration of complex systems or networks based on the potential benefits and/or consequences.
• Analyze, evaluate, present and recommend technology methodologies and solutions.
• Solve problems with minimal disruption or impact to the organization business functions.
• Determine the systems or networking requirements to plan for or enhance the core data processing needs for the system or network.
• Develop technology guidelines and standards.
• Develop alternative solutions within established guidelines and standards.
• Direct, schedule, and orchestrate the installation, maintenance, and enhancements of complex IT systems and networks.
• Prioritize competing requests for service.
• Develop policies, procedures and standards relevant to the area of expertise.
• Interpret and implement IT policies, procedures, and standards.
• Plan, schedule, make work assignments, set priorities, train, mentor, evaluate, and select staff;
• Operate modern office equipment including computer hardware, software, and internet and web-based applications.
• Compile data and prepare narrative and statistical reports.
• Appropriately respond to requests and inquiries from the public, State Bar members, or other departments.
• Communicate clearly and concisely, both orally and in writing.
• Establish and maintain effective interpersonal relationships at all organizational levels as well as with the public.
• Maintain the confidentiality of information.

Minimum Qualifications

Education:

• Bachelor's degree in computer science or a closely related field that develops skills related to the essential duties, or equivalent academic achievement.

Experience:

• Five (5) years of experience at the advanced journey-level in the information systems field including system analysis, business process design, development and implementation of business application solutions or IT project management, of which at least one (1) year was in a lead capacity.

License, Certificate, Registration Requirements:

• Possession of one or more approved information technology certificates and/or completion of other approved technology-related training may substitute for some or all of the required education.

Applicants who meet the Minimum Qualifications will be required to successfully complete the State Bar's selection process for this classification.

About the State Bar

The State Bar of California's mission is to protect the public and includes the primary functions of licensing, regulation, and discipline of attorneys; the advancement of the ethical and competent practice of law; and support of efforts for greater access to, and inclusion in, the legal system.

Our Values

Clarity | Investing in Our People | Excellence | Respect | Growth Mindset

Learn more about our values.

DEI Statement

We are a diverse, equitable, and inclusive workplace where all of our employees and prospective employees experience fairness, dignity, and respect.

Learn more about our commitment to DEI.