From Software Engineer to AI Engineer: Lessons from Janvi’s Journey Through 46 AI Startups to OpenAI
The explosion of AI startups and advancements in large language models (LLMs) has created a dynamic and sometimes overwhelming landscape for engineers aspiring to work in AI. Janvi, a software engineer turned AI engineer, shares her unique journey of interviewing at 46 AI companies, learning the nuances of the AI job market, and ultimately landing a role at OpenAI. Her story offers valuable insights for anyone interested in AI engineering, navigating startup culture, and evolving alongside emerging technology.
Understanding the AI Landscape: Product, Infrastructure, and Model Companies
Janvi categorizes AI companies into three segments to clarify the sprawling AI ecosystem:
-
Product Companies: Build applications on top of AI models, such as Coda, Cursor, and Hebia. These companies focus on delivering end-user functionalities powered by AI.
-
Infrastructure Companies: Provide the tools and platforms that enable product companies to effectively use LLMs. Examples include inference providers (Modal, Fireworks), vector databases (Pinecone, ChromaDB), and observability tools (Braintrust, Galileo).
-
Model Companies: The creators of the AI models themselves, including giants like Google and Meta, as well as specialized companies like OpenAI and Anthropic.
This framework helped Janvi narrow her focus to model and infrastructure companies to broaden her skills beyond her previous product-focused experience.
The Early Journey: Internships at Google and Microsoft
Janvi’s internships at Google and Microsoft were pivotal. Without personal connections, she applied through portals and stood out through her essays and projects built outside class. Preparation for these roles involved deep study of classic coding interview materials, like “Cracking the Coding Interview,” long before the popularity of LeetCode and Blind 75.
Her Google internship exposed her to large-scale codebases and best engineering practices like unit testing, while Microsoft allowed her to dive deeper into operating systems, specifically on Azure OS. Importantly, she learned the value of expressing preferences during internships, which can lead to more fulfilling work experiences.
Choosing Startups Over Big Tech: A Strategic Decision
Despite having offers from Google and Microsoft, Janvi chose to join a startup, Coda, seeking breadth and rapid professional growth. Startups provided her opportunities to ship code frequently, tackle zero-to-one problems, and gain non-technical skills like product management and business understanding.
Her criteria for selecting startups evolved over time, focusing on:
- High and steep revenue growth
- Large addressable markets
- Loyal, obsessed customers
- Competitive advantages ensuring the company’s success
She emphasizes doing thorough due diligence about startup viability, including revenue, margins, and customer feedback, often gathering this information from public forums, direct customer conversations, and even investors.
Transitioning Into AI Engineering at Coda
When AI technologies like ChatGPT emerged in late 2022, Janvi proactively learned deep learning foundations, from tokens to transformers, through self-study and hackathons—even after being initially declined to join Coda’s AI team.
Her persistence paid off: by demonstrating her passion and skills through independent projects and hackathons, she secured a spot on Coda’s AI team. She highlights the importance of building intuition around these technologies and learning by doing through hackathons, which also helped her understand production challenges of integrating stochastic AI models.
What Does an AI Engineer Do?
Janvi describes AI engineers as software engineers who build on top of models, involving:
- Experimentation with models and tools
- Prototyping solutions to real customer problems
- Transitioning prototypes into production systems
The role combines traditional software engineering with domain-specific tasks like prompt engineering, fine-tuning models, and evaluating model performance. For example, running evaluation suites can incur real costs, unlike traditional unit tests, adding new dimensions to engineering discipline.
Favorite AI Project: Workspace Q&A at Coda
Janvi’s proudest project at Coda was building a chatbot leveraging retrieval augmented generation (RAG) to answer questions about users’ workspace documents. This prototype evolved into “Coda Brain,” a product demoed at Snowflake Dev Day and later expanded by a larger team.
Her experience underscores a key lesson: don’t wait for permission to explore new technologies. Taking initiative and continuous learning can accelerate career growth, especially in emerging fields like AI.
Interviewing at 46 AI Startups: Market Observations and Strategies
Over six months, Janvi interviewed extensively across product, infrastructure, and model companies. She noticed:
- AI startup teams are lean, fast-moving, and mission-driven.
- Evaluating startups requires understanding unit economics, especially for infrastructure companies with expensive GPU costs.
- Model companies have to stay ahead of open-source alternatives to justify premium pricing.
- Due diligence is crucial; if you’re not excited about a company or lack transparent information, it’s better to wait.
Her interview preparation balanced traditional coding and system design (utilizing resources like NeetCode and Alex Shu’s books) with project-based interviews, which allowed her to showcase passion and practical skills.
Working at OpenAI: Speed, Scale, and Safety
Janvi now works at OpenAI on the safety team, focusing on:
- Building low-latency classifiers to detect harmful model outputs
- Measuring real-world harms and mitigating risks from model misuse
- Integrating safety mechanisms across products
She highlights OpenAI’s unique combination of startup-like speed and massive scale (handling 60,000 requests per second), alongside an open culture that fosters learning and collaboration. Engineers are trusted to ship fast with minimal bureaucracy, emphasizing ownership and impact.
Surprising Realities of AI Engineering
Janvi shares that AI engineering often involves building temporary solutions to current model limitations, only to scrap and rebuild as models improve (e.g., evolving from custom JSON parsing to function calling and then to the MCP paradigm). This requires adaptability and a mindset of continuous iteration.
The Future for New Graduates and AI’s Impact on Engineering
Contrary to fears that AI will replace junior engineers, Janvi believes AI empowers all engineers to focus on higher-level creative tasks. The key skill will be knowing when to rely on AI and when to deeply understand system internals, especially for robustness and debugging.
She stresses that curiosity and understanding the “why” behind technologies remain critical traits, as engineers must still design and maintain complex systems. AI tools accelerate productivity but don’t replace the need for strong foundational knowledge.
What Remains Constant in Software Engineering?
Despite AI’s transformative impact, core software engineering fundamentals endure:
- Designing high-level architectures
- Debugging complex systems
- Writing maintainable, well-structured code
Janvi finds value in revisiting classic software architecture books like The Mythical Man-Month and Software Architecture by Mary Shaw and David Garlan to uncover timeless principles that still apply.
Practical Tips and Final Thoughts
- Be proactive: Don’t wait for permission to explore AI—start building side projects or join hackathons.
- Do your homework: Research startups thoroughly before joining; talk to customers, investors, and read industry analysis.
- Balance learning and building: Use AI to accelerate coding but also deepen your understanding for ownership.
- Embrace change: AI engineering requires agility as technologies and best practices evolve rapidly.
- Cultivate curiosity: Ask “why” and seek to understand underlying mechanisms, which leads to better engineering.
Janvi’s journey—from learning transformer basics on her own to building impactful AI products and joining OpenAI—illustrates that dedication, curiosity, and strategic decision-making can open doors in the fast-paced AI industry.
Resources Mentioned
- Cracking the Coding Interview by Gayle Laakmann McDowell
- NeetCode and Blind 75 coding practice
- Alex Shu’s System Design books
- Hackathons like Buildspace and internal company events
- Blogs, Twitter, and open source documentation (e.g., LangChain)
- Software architecture classics: The Mythical Man-Month and Software Architecture by Mary Shaw and David Garlan
Conclusion
Janvi’s story is a powerful testament to self-driven learning, thoughtful career choices, and embracing emerging technologies. For engineers aiming to transition into AI or grow within the AI ecosystem, her experience offers a practical blueprint: understand the landscape, continuously build and learn, and don’t hesitate to take initiative. The AI revolution is still unfolding, and there’s room for passionate engineers to shape the future.
For more deep dives on AI engineering and insights from OpenAI teams, check out the Pragmatic Engineer podcast and related resources.