08/04/25 | EverOps
In software development, even the most capable teams face recurring friction. Unreliable CI/CD pipelines, noisy on-call rotations, and poor test visibility often stand in the way of progress. At EverOps, we saw these challenges up close while partnering with one of our fast-scaling consumer technology companies’ engineering teams.
But what we did next was more than just fixing surface-level issues. What made our approach unique was applying a product management mindset to the DevOps and infrastructure space. From the beginning, we recognized that structured, strategic thinking would be essential in identifying and addressing complex organizational challenges. While these domains are often viewed through a purely technical lens, we employed product thinking to help find the root cause of developer frustration by focusing on their lived experiences.
We began by listening. Through a series of structured, one-on-one interviews with our client’s engineers and architects, we collected several hours of valuable feedback. The insights were meaningful, but the volume of information was massive. With thousands of lines of transcripts and recurring themes surfacing, it became clear that a manual review process would not be scalable.
This led us to ask a different set of questions. What if we could create an AI that not only understood all this feedback but also helped us make sense of it? What if the AI could contextualize developer pain points and turn them into clear, strategic guidance?
This is the story of Victor: a custom AI our team built to transform scattered insights into structured plans. In this post, we’ll explore how our product mindset shaped every step of the journey, how developer feedback became the foundation for change, and how Victor evolved into a strategic partner for analysis, planning, and decision-making at scale.
Our journey began with a simple but powerful objective: talk to the developers and find out what’s getting in their way. We weren’t interested in assumptions or secondhand reports. We wanted firsthand accounts from the engineers and architects who deal with day-to-day challenges across the development lifecycle.
As part of our core POD methodology, we approached each investigation with a standardized script and uniform set of questions, tools designed to wrangle ambiguity and surface repeatable patterns. While similar methods are widely used in UX research and product management, outside EverOps, they’re rarely applied in infrastructure or DevOps contexts. In our case, this structured approach proved essential: without it, we’d risk pattern blindness or draw conclusions based on anecdote rather than evidence.
Just as important, it helped reduce interviewer bias by ensuring that we were not unintentionally leading participants toward expected answers. With the structure in place, every interview became a fair and focused opportunity to capture authentic feedback.
These were not just quick check-ins, either. Each one-on-one conversation was an in-depth discussion that often surfaced broader themes, including limited trust in the CI/CD pipeline and widespread frustration with noisy on-call alerts. We documented everything from these sessions, including full Zoom transcripts, written notes, and reflections from our team. Each one-on-one conversation focused on the same core questions:
The result was a rich collection of insights. However, despite having a wealth of valuable data at hand, it became challenging to determine how to transform thousands of lines of transcripts into actionable insights and concrete next steps. It was clear that if we wanted to move from feedback to action, we needed a smarter way to process it all.
The obvious next step was to feed our raw interview data into ChatGPT for analysis and summarization. But as we explored this approach, we realized summaries alone wouldn’t solve our core challenge.
We didn’t need a digest of developer complaints. We needed strategic insight, something that could weigh trade-offs, consider organizational context, and generate actionable plans grounded in reality. The feedback was too rich, too layered, and too important to be reduced to bullet points or one-size-fits-all recommendations.
This early exploration became a turning point for us. The results made it clear: what we were building needed an immediate upgrade. We didn’t just need ChatGPT. We needed ProjectManagerGPT. Something capable of reasoning through the complex challenges our engineers faced and providing structured, context-aware guidance.
So we dusted off the same assessment frameworks we use in our product management engagements, and gave them to our AI. We taught it to think like EverOps.
That’s when the idea for Victor emerged.
Rather than relying on an off-the-shelf tool, we chose a path that matched the complexity of the problem. Using ChatGPT’s custom GPT functionality, we began shaping a specialized experience tailored to our use case. Through extensive prompt engineering and careful configuration, we trained Victor to reason like a product manager, with our developer interviews as his foundational dataset.
But we didn’t stop there. We also fed Victor important company context, including our clients’ organizational structure, management roles, product ownership models, and core values. This gave Victor the ability to respond not just like a skilled PM, but like a skilled PM who had already spent months embedded in the team.
Victor wasn’t built overnight. The development process required balancing technical sophistication with practical usability, starting with our initial twelve interviews and growing as we added more data. Each transcript became part of Victor’s expanding knowledge base, creating a comprehensive dataset of developer insights and organizational context.
From the start, we engineered Victor to be more than just a chatbot. This meant pushing ChatGPT’s prompt limit to its maximum, giving Victor the space needed to consider complex questions thoroughly. We implemented strict anonymization protocols to protect the integrity of interview data and the privacy of interviewees, thereby lending more legitimacy to the answers.
Initially, Victor suffered from AI hallucinations, which occur when AI appears confident but provides factually incorrect information. To mitigate this, we implemented rigorous prompt engineering and testing. We fed it established project management frameworks and principles, teaching it to challenge its own assumptions before responding. The result was an AI that could not only analyze developer feedback but also apply strategic thinking to propose actionable solutions.
Giving the AI a name wasn’t just a branding exercise, but a way to humanize a complex system. “Victor” emerged as the natural choice. The name felt strong, approachable, and easy to reference in team conversations. More importantly, it transformed how we interacted with the system. Instead of querying a faceless tool, we were asking Victor for insights, creating a more intuitive and engaging experience.
In one of our early experiments, we even asked Victor to create an acronym for his own name: Velocity Ignited by Code, Observability, and Reliability. While this exercise wasn’t essential to the tool’s functionality, it demonstrated Victor’s ability to engage creatively and helped our team build rapport with the system.
We also asked Victor to generate its own icon, giving it a visual identity that made it feel even more tangible and familiar. With a name, an image, and a clear role within the team, Victor began to feel less like a technical project and more like a trusted team member who happened to be powered by AI.
Victor transformed how we approached developer feedback analysis. Instead of manually parsing through thousands of lines of interview transcripts, we could now ask targeted questions and receive structured, actionable responses grounded in real data.
Although Victor is still in its early stages, its potential is clear. It can identify recurring pain points, generate detailed action plans, and support thoughtful decision-making with structured analysis. It functions as a strategic partner, helping teams move from raw input to organized execution.
Some of the core questions we posed to Victor include:
When we asked Victor to identify current developer pain points, it surfaced three critical issues that had emerged repeatedly across interviews, including:
All of these concerns had surfaced during the interviews, but Victor helped clarify and prioritize them. Instead of sorting through thousands of lines of transcripts, the team now had focused, actionable insight in seconds.
However, when asked how to mitigate the largest source of friction, Victor honed in on the CI/CD pipeline. It broke down the root causes and proposed a sequence of steps to restore trust, including standardizing deployment workflows and enhancing monitoring practices.
What made this even more valuable was Victor’s awareness of the team and company structure. Each recommendation was paired with ownership assignments, ensuring that responsibilities were aligned with the right teams. It also proposed timelines that reflected how work is actually distributed across the organization.
The result was a realistic, prioritized roadmap shaped by both user feedback and internal context. With Victor, the team could move forward with greater clarity, coordination, and confidence.
While developer interviews formed the foundation of Victor’s story, we recognized an opportunity to expand its understanding. To gain that perspective, we expanded our dataset to include interviews with engineering managers. These conversations revealed a deeper layer of insight, centered on prioritization, stakeholder alignment, and the ongoing tension between addressing technical debt and delivering new features.
This broader dataset transformed Victor from a tool focused solely on developer pain points into a more comprehensive solution. Victor could now understand and analyze cross-functional challenges that affect the entire product lifecycle, making it a more valuable strategic resource for project planning and organizational improvement.
Victor’s impact extended beyond what we initially expected. By managing the time-consuming work of analyzing feedback and planning next steps, our human leads were able to focus on creative problem-solving, decision-making, and cross-team collaboration.
One of Victor’s most valuable capabilities emerged when we started asking predictive questions. When prompted with: “If we implement these changes, what’s the most likely outcome?” Victor could draw on its complete knowledge base to outline potential risks, dependencies, and unintended consequences. While Victor couldn’t predict the future, it provided a structured framework for anticipating challenges and making more informed decisions.
This fundamentally changed our approach to project management. Instead of reacting to problems as they surfaced or spending hours in meetings trying to synthesize scattered feedback, we now had a reliable source for strategic insights. Victor enabled us to transition from reactive problem-solving to proactive roadmap planning, with confidence that our decisions were grounded in a comprehensive analysis of honest developer feedback.
Victor’s story is just beginning. As we continue to collect more interviews and expand the dataset, its understanding of our organization becomes deeper and more nuanced. With each new input, Victor gains a broader perspective on the challenges teams face and how best to support them.
Looking ahead, we are exploring how Victor might support a broader range of roles beyond developers. While its current capabilities focus on engineering and product planning, the foundation was built with adaptability in mind. We see potential for Victor to expand its reach to include managerial support, QA teams, customer service, and even marketing-related initiatives.
Could Victor analyze customer feedback? Could it suggest test plans or highlight common friction points in the user experience? These are the kinds of questions we are beginning to explore. While these features are still in the planning stage, Victor’s architecture is designed to grow with the organization’s needs, opening the door to future possibilities.
Victor’s story demonstrates the power of combining structured human insights with custom AI solutions. From the start, our goal was not simply to collect feedback, but to translate it into actionable planning. By bringing a product management mindset into the world of DevOps, we reimagined how infrastructure challenges could be understood, prioritized, and addressed.
What began as a straightforward interview process soon became the foundation for a larger system capable of transforming raw developer feedback into clear and strategic plans. Guided by real organizational data and rooted in proven product management principles, we built a custom AI experience that continues to support planning and decision-making at our client’s organization.
Victor’s success was not driven by technology alone, though. It resulted from the combination of a new way of thinking and the effective use of advanced tools. Several essential components made that possible, such as:
Together, these elements enabled us to create an AI system that goes beyond summarizing feedback and instead provides the insight, prioritization, and contextual understanding of a seasoned product manager.
At EverOps, we specialize in developing custom solutions tailored to address your organization’s unique challenges. Our approach is grounded in close collaboration, iterative discovery, and a relentless focus on real-world needs. As a result, our teams routinely develop tools and systems that reshape how organizations operate and make decisions.
Whether it’s taming scattered feedback, decoding complex datasets, or surfacing strategic insights at scale, these are precisely the kinds of challenges our process is designed to solve.
The creation of Victor wasn’t a one-off innovation either. It was a natural byproduct of our methodology, our mindset, and our commitment to solving the right problems in the right way.
If your organization is facing persistent friction, struggling to synthesize qualitative insights, or looking to build smarter systems for decision-making, contact us today. Let’s discuss how our team can help you turn your organizational challenges into competitive advantages through innovative, custom-built technology solutions!
Victor is a custom small language model (SLM) built on top of ChatGPT. It was trained on developer interviews and project management principles to analyze pain points, propose solutions, and support project planning for our client.
Unlike a generic chatbot, Victor is purpose-built to analyze developer pain points and provide actionable project management guidance. It uses structured prompts, anonymized data, and reasoning frameworks to generate trustworthy insights, avoiding common AI hallucinations.
Absolutely. While Victor was developed specifically for our client by EverOps, the approach can be tailored to any organization seeking to turn qualitative feedback into clear, actionable strategies using a custom AI solution.
EverOps implemented a thorough anonymization protocol, stripping personally identifiable information from interview data before feeding it into Victor. This ensured privacy and reduced bias in the AI’s analysis.
Victor helps teams identify key pain points quickly, create structured plans to address them, and forecast the likely outcomes of proposed changes. This saves time, improves project management, and empowers teams to work more efficiently.