[Remote] IDF Dataflow Analyst
Note: The job is a remote job and is open to candidates in USA. Spatial Front, Inc is seeking an IDF Dataflow Analyst to support a large-scale data modernization initiative. The role involves managing the flow of real-time data, assisting in the migration of datasets to a cloud platform, and ensuring the reliability of data delivery across environments.
Responsibilities
- Support the ingestion and migration of large-scale datasets into a cloud-based data platform
- Assist in modernizing legacy dataflow pipelines into automated, cloud-native solutions
- Maintain high availability and reliability of real-time data delivery across hybrid environments
- Monitor, troubleshoot, and optimize data ingestion and routing processes
- Manage the end-to-end flow of real-time data, including validation, quality control, archival, and retrieval
- Execute and support ingestion processes for mission-critical datasets into the cloud environment
- Assist in transitioning legacy workflows into API-driven and automated scripting solutions (Python/Bash)
- Ensure continuous data flow in a 24/7 environment; troubleshoot and resolve data delivery issues
- Monitor and optimize data routing applications and cloud services to ensure performance and low latency
- Maintain and enhance scripts and legacy tools (Bash, Perl, SQL) while supporting migration to modern solutions
- Support automated deployment, testing, and monitoring of dataflow applications and pipelines
- Serve as a technical escalation point for complex data ingestion and formatting issues
- Develop and maintain technical documentation, processes, and configuration standards
- Participate in a 24/7 on-call rotation to support mission-critical operations
- Work with internal teams and external partners to resolve data access, formatting, and integration challenges
Skills
- Bachelor's degree in Computer Science, Information Systems, or related field (or equivalent experience)
- 5–8 years of experience in a technical, data, or systems support role
- 1–2 years of hands-on experience with cloud platforms (AWS, Azure, or GCP)
- Strong scripting experience in Python and Bash (Perl experience a plus)
- Experience supporting data pipelines, ETL processes, or system integrations
- Familiarity with CI/CD tools and automated workflows
- Minimum 2 years of Linux/Unix system administration or support experience
- Ability to obtain and maintain a Public Trust clearance
- One foundational or associate-level cloud certification (e.g., AWS Certified Cloud Practitioner or equivalent)
- Meteorological Domain Expertise: Experience working with scientific data dissemination and observation systems within NOAA, NASA, or Department of Defense environments; familiarity with standard meteorological data formats such as METAR, TAF, GRIB, and BUFR
- System Monitoring: Hands-on experience with enterprise monitoring tools, including Big Brother, Nagios, or cloud-native solutions (e.g., Amazon CloudWatch) to ensure system performance, availability, and reliability
- Database Management: Proficiency in SQL and a strong understanding of relational database design, querying, and maintenance
- Version Control: Experience using version control and change management tools such as Git, Subversion (SVN), or similar platforms
- Desired: Advanced cloud certifications
Benefits
- SFI offers a comprehensive benefits package. Details available at SFI Careers.
- This is a full-time, W2 position.
Company Overview