I got to work on whatever I wanted for my entire career. This is Carrie Notchenberg. He's a cyber security expert who is a fellow at Semantic, which is four levels higher than staff. I look for things with big business impact. I look where there were gaps. From there, he joined Google X as a principal engineer and fought imposter syndrome. And what if like I'm not good enough for Google or Meta or something. As a professor at UCLA, he's seeing firsthand how AI affects the classroom. I allowed students to use LMS. In retrospect, you know, that was a bad idea because I think they were using it in ways that hindered learning. How do you even tell if they're using LMS, though? Well, the one way you can tell is I learned a lot from his career stories and hope you do, too. I get this call, a frantic call from UCLA. Can you still teach? Because our lecturer bailed on us. I have PTSD from that experience, I have to say. Before we get into all the juicy parts of your career like working at Google X or autonomous vehicles with Lyft, I want to start by kind of laying the high level groundwork. So you worked at Semantic for a long time and became their senior most, you know, similar to a chief scientist type of role. Kind of want to go over that and all the lessons you might have learned there and you know, anything interesting that came up there. So could you share the highlevel story arc of you working at semantic and how you grew to fellow? Totally. So I actually started back in I think 1992 as an intern at Peter Norton Group. And so Peter Norton Group was later an acquisition by Semantic but some of your viewers are going to remember Norton antivirus from Norton and Norton Utilities and so on. And so um I think I was the first intern at Norton at Peter Norton computing and they didn't even have a desk for me. So, I literally worked in a QA lab with the QA lab manager who was a guy that used to sell knives for a living because like back then in 1982 they weren't professionally trained software people, right? So, basically they would just get whoever knew something about computers and this guy knew something about computers to run the lab and I would work in one of the test computers. Um, and so that was my first, you know, summer internship. uh and then fast forward uh when I left in 2016 I had had become the senior most engineer of the company. So from the junior most person the first intern to the senior most person at semantic which would had acquired Norton. So it was a long ride but that that's the high level. Was cyber security something that really you were focused and really wanted a job in cyber security or it just by chance? Yeah. So at the time when I was working at Peter Norton Group, I wasn't working in cyber security. I was working on something called Norton Commander, which was like a file utility manager. So I didn't actually work on cyber security until my third year, my third year of internships uh at this point at Semantic where they they had acquired a product called I don't remember what it was, but it was an antivirus product that they renamed Norton antivirus. So that product was uh acquired and rebranded uh and they had a team of people analyzing computer viruses. So my third year of internship that's when I got into cyber security. I had no experience. They just needed an intern and they threw me on it. Like what is the the career ladder look like? Because I think Google and those companies came in at some point and said, "Hey, this is L3. This is L4. This is L5." And then kind of a lot of companies copied that. At Semantic, what did career progression look like? I would say the levels generally track to companies like Google and Facebook or Meta or going all the way from like a junior you know software engineer through software engineer senior software engineer staff and so on. So um the levels are very similar. The difference was for me at the very high levels of distinguished engineer and fellow at semantic you know when I went to Google they basically said you know we can't just hire fellows you know we don't haven't experienced you we don't know if you're going to do well here so effectively I was downgraded to a st to a principal engineer level eight from what would have been probably a level 10 at semantic it was a vice president role at semantic What does the hiring process even look like for people so high level? Um, was that a tailored bespoke process? When I went to uh Google, basically the former chief operating officer of Semantic, guy named Steven Gillette, had recently transferred into Google X and they were starting a stealth project in Google X, which we could talk about. he had known me from my time at semantic and basically you know said hey why don't you talk to the team and see if there's a good fit and so I didn't have to apply I was just you know brought in and and uh we had a one meet and greet with the comp the team's founders uh in Venice and then we had an interview a day's worth of interviews probably about eight interviews and you know six weeks later I had my offer letter so you know for a lot of software engineers that are in the lower The interviews are leak code there or system design and there's you know some generic behavioral interview for at the high levels. What what does that loop even look like? Is it is it mostly behavioral? Do are they asking you leak code stuff too? So I don't believe that I had any coding problems during that interview. I if I recall correctly there were some design there was one design problem. Um but mostly these were leadership-l like questions like you know how would you solve a a hard problem or how did you solve a hard problem in your career um how do you deal with conflicts um you know tell me your thoughts on you know where the field is going you know Google X is a forward-looking or X rather is a forward-looking organization and so uh some of the discussions were focused on that um some of the discussions were lit literally sales pitches like I didn't really have to say they went and talked to me about where they thought the world was going to try to get me excited, right? So, I think it probably depends on the person, but in my case, I think they thought there was a good fit and it was really about sort of going through the motions of, you know, having interviews and selling me on the on the role. Okay. So, then going back to Semantic, I mean, you know, g getting promoted to fellow, what do those highest level promos look like? At least at semantic, most promotions up through what was called technical director or senior technical director um which would be the equivalent of let's say staff engineer or senior staff engineer probably at other companies. Most of that was done just within the organization. So as long as a vice president or senior vice president was okay with a promotion, it was it was allowed. uh at higher levels for distinguished engineer and fellow. It was there was a a basically um core group of technologists led by the CTO of the company. We would meet once a quarter. Um we would get applications from those people and we would then review their applications and actually have them come talk to us about their work u and then make a decision based on that. And so the reason that we did it that way is that we'd have consistency across all divisions, all teams, uh, of what it meant to be a distinguished engineer or or a fellow for the organization. Um, so that's the process we went through and it wasn't just per division, it was for the company. So when you got promoted to fellow, was that one of the happiest moments of your career or was it something you expected, not a big deal? You know, it was very interesting. So I did not have to go through that process to get promoted to fellow. So I was a distinguished engineer at Semantic. Um and they had that process. I wasn't part of it because I wasn't a distinguished engineer at the time. That that team is made the team of that does promotions is made up of distinguished engineers at the time. We had acquired a company called Veraritoss and Veraritoss had a notion of a fellow which Semantic didn't. So basically after the merger between the two companies, they said okay we need to level set the role because the people at Semantic have never had this kind of role who would belong there and I was nominated without my knowledge and it just happened and I got a congratulations or maybe my boss filled me and said hey there's some discussions going on thank you know and then you're a fellow. So they basically took my port portfolio of stuff and gave it to the the team that did the evaluations and that was the end of that. I mean with your growth to to fellow something about the way you work sets you apart from other engineers. What do you think are those things for you? I think what helped me get to fellow was working on really impactful projects for the business. not necessarily always the most technically difficult projects although many of them were but more impactful like they move the needle for the company. I had so many of of them under my belt at that time that they just said you know the criteria are this many projects should be done of the of this scope and I had plenty more and so they basically said you're you know you're above the bar. So what was the what was the key insight to doing those projects is finding things where there where there were gaps that you know things the company needed where people weren't stepping up to do them and you know I did them maybe a quick step back at Savant it was sort of like the wild west there wasn't an engineering culture per se there was just people working on stuff and like projects would often run really late because you know it was a little bit more of a wild us. We didn't have an agile process. Um, we didn't really do our own unit an integration task like we'd throw over our code to QA and they would worry about it and we'd work on the next thing. And in fact, I had very unusual circumstances. I didn't rise to a path where I was given a small project and somebody said, "Okay, do this. Here's here's a box around the project." They basically said, "Carrie, you already sort of know the technology because you've been working on it as an intern. Go figure out what to do and do it." which was amazing because here I am like a junior software engineer and I got to work on whatever I wanted for my entire career at semantic never assigned work to do. Yeah, it was crazy, which was great. And I, you know, and so I got to just pick projects that I thought would be impactful. And I picked well. I got I guess I picked well because I, you know, project after project landed. They were integrated. We did tech transfers into the product, they shipped. So that's how I got there. It wasn't like, oh, I did increasingly bigger scope projects that somebody gave me. So how'd you train that that project taste? Because impact typically leads to career growth everywhere. So yeah. Yeah. And it's a good question and it's I look for things with big business impact. I look where there were gaps. So like way back when you know this is like old news now but it's sort of interesting like we had um these computer viruses which are emerging which were called polymorphic viruses. They were selfmutating malware and they were selfmutating in a way that they there could be like literally quadrillions of variants and the way that the teams were working on these things but back when I was an intern even is they would basically write uh handwritten assembly language to go look for telltale signs of a variant you know in the mutation. Okay. And the problem with that is it worked really great except it took six months because we shipped a new product every six months. And so it took six months to discover to handle a virus and then the next day somebody released three new viruses which were slightly you know slight variance or differences, you know, different viruses but mostly the same. And then all of that work would not work anymore. And so, you know, I would look at that problem. I'd say, "Oh, there's a need to be able to move more rapidly in covering new malware. And by the way, detecting these selfmutating threats is not like using a rag X. You can't just, you know, search for a string to find these things." So, I'm like, "Oh, that seems like a really interesting hard problem." So, I picked it and then I started working on it. That was my master thesis and then eventually transferred to the product. If you were to think about the things that you took on, they were a series of I guess side projects or or you know, whatever you wanted to take on where you you'd take on this new thing, maybe something else would come in, you take that on. Is that kind of how you were working? I would say there are probably six or seven times in my career where I'm like, "Oh, the company needs this type of thing. Let me go spend six weeks, two months, five months figuring out what that looks like, building prototypes, talking to engineers, and figuring out what they need. and then you know building that and there were other times which is probably 80% of my career where I was just sort of tweaking those things in other words we built them we were trying to either tech transfer it so I was helping with that fixing bugs improving those were sort of incremental improvements on those systems but that that it's one of those two things generally so you mentioned a little bit about viruses and when I was doing some research I saw that you had done some storytelling on top of stuckset and kind of compiled I think that's such an interesting story. Uh can you tell me a little bit about stuckset? Maybe we can go into that. Sure. Stuckset was um at the time just unfathomable. It was just a very complex piece of malware which was multiplatform. So it didn't just infect like Mac machines or Windows machines. It infected I think you know Windows machines but also like microcontrollers that would actually run like centrifuges and so on. Um, and so it was probably the first multi-platform piece of malware we had discovered. Uh, use zero days in order to break into systems that you know that you know basically vulnerabilities, exploiting vulnerabilities that hadn't been patched because they weren't even known about. And it didn't use just one of those or two of those. I think it used like six different vulnerabilities to spread, many of which were zero days. My god. Um, it would literally stealth itself. So on your computer, if you were to look at a at a thumb drive which had Snapchatnet on it and look in your your your Finder application or your Windows, you know, file system application, you would see, you know, nothing there, but it was there. You'd stick that in your computer, it would auto launch. It actually had a a payload to auto launch. If you were to look at the logic that was running on a centrifuge or rather the controller that ran the frequency converters, you would not see any of the specs in that logic. It was in that controller. But if you downloaded the the logic from that controller onto a Windows machine, it would stealth and remove the logic from stuckset as it pulled it off. And then if you updated that logic, for instance, it would reinsert itself into that logic to reinfect it as it went back. So we actually like sort of piggyback on back and forth stealth itself. Um it was just amazing. And then of course how it disrupted the centrifuges is super interesting as well. Yeah, it's so complicated and sophisticated that it makes me wonder who wrote it and I saw something like it was, you know, 50 times bigger than the average virus. Incredibly complicated software. And I was reading into Wikipedia a little bit before we kind of it said no one has claimed credit for who wrote this thing. Who do you think wrote this thing? It's I think it's pretty good. You'd be pretty safe to say it was the Israelis and the American government. You know, my understanding or recollection is that there are water not watermarks but sort of, you know, coding styles or things in there that sort of implicate both uh governments. Have you ever looked at the source code or played? I have not. I didn't do any analysis on stuck set. Um my career was focused early on analyzing malware like literally looking at the machine language and disassembling and so on. But later on in my career it was mostly about detecting sort knows how how could I build algorithms to detect that malware rather than hands-on analyzing the malware myself. So I'd never looked at stuckset. Yeah. You mentioned a little bit about uh assembly code. Did you ever write assembly code when you were working at cement? I did. Yeah. I wrote assembly code as an intern. Um and uh although back in those days it was mostly C. Yeah. Um but some assembly as well. And I remember the first antivirus engines were written in assembly for speed. And one of my first tasks as I joined full-time was I said, you know, this really needs to be a C so it's more maintainable. So we ported the thing to C and actually made it faster because the people back then people didn't know algorithms. They didn't understand what an what a big O was or how to you know they would do linear searches. And so we were able to go and take something in assembly language, move it over to C, have less code, um, and it would be, you know, five times faster. So I see. So the the speed ups moving from assembly to C was due to better algorithms and things like that. It wasn't because of a compiler or something. No, the compilers weren't that great back then. But even without an optimizing compiler, if you use a hasht versus or binary search versus a linear search over 60,000 signatures, you know, I I saw that you worked at Semantic for a long time and you know, I think in the tech industry, it's common for people to move around here and there. What do you think kept you at semantic as long as you were? You know, that's a great question. Um, if I have to be perfectly honest, I would say imposttor syndrome. Really? So well yes and no. So at semantic I didn't really have imposttor syndrome because I had done a lot of stuff and I was well regarded you know I was known in the company and so I had a good safe place but I always worried what if it just is because I'm at Semantic and I grew up here and I learned the stuff here. what if I went somewhere else and I wouldn't be able to learn the stuff or what if people had different standards and what if like I'm not good enough for Google or Meta or something and so I stayed because it was comfortable and I complained I complained all the time I wasn't happy later on in my career I have to be honest with you I wasn't doing things that made me happy more when you get more senior you do a lot more BS right and and and you also have the opportunity not to do as much BS but you have to push yourself not to do it because it's very easy to, you know, go to meetings and, you know, have broad discussions and it's not really that necessarily fun, right? Um, and so I wasn't happy near the end of my tenure at Semantic, but I was afraid that I wouldn't be able to do well or I'd fail the interview process. And so I just stayed and it was comfortable. throughout your career there were so many promotions and you had so much impact for someone like you to have imposter syndrome you know I feel like that shows that a lot of people you know it's it's a very natural feeling for a lot of people did you eventually you did leave semantics so was there anything that helped you uh overcome imposter syndrome you know what the thing that that helped me was that somebody said hey we want to interview you we think you'd be a good fit and so I said you Well, I'm probably going to fail this interview. I'm sure I'm not good enough, but I'm going to do it. And so, I just did it. And so, that, you know, I needed an external pull or push, I don't know what you would call it, but in order to get me to to take the chance, and then it worked out. But for me, like in my head, I was, you know, I wasn't competent to do that job. You know, you you mentioned uh also that at the highest levels, there's uh, you know, a lot of BS and, you know, I guess it sounds like meetings and things like that. Do you have any uh I guess tips on how to be less involved in the BS because I think that's a natural pull pull for anyone. Yeah, it's sort of natural definitely it depends what you're doing. I mean some some tech senior technical directors and distinguished engineers even fellows were working dayto-day and building code and and working with their teams. It just depended uh I was an individual contributor vice president. So I was an IC through my entire time at semantic. other people would actually manage teams and work uh more closely on projects. You know, it's just it's inevitable, right? In other words, you're having more strategic meetings and then the problem is you're having a strategic strategic meeting with a bunch of people, many of which many of whom don't necessarily know that much, but they have an opinion because everybody has an opinion. Um, and there's a lot of debating and a lot of arguing and a lot of like, you know, posturing for, you know, for power. And, you know, it's just there's a there's a lot of garbage that comes with being more senior, unfortunately. Like, there was some some joy, especially for me when I got to pick my own projects to be able to just sit down and literally go two months with nobody asking me what what are you doing? And, you know, I'm just like cranking and trying things. That doesn't work, but that does. And super exciting. Right. Right. Then you get into a room with seven people and you're like, "We've agreed that this is our new company strategy." One of my last rule uh things of the company I did was actually define the company technology strategy for the whole company. And everybody had agreed to it. The CEO had agreed to it. And then we get in a room and everybody would say, "Oh, sure. But you know, we have to make money on our projects or products." And so, you know, adding those features to align with technology strategy that's gonna set us back. And we've been told we have to make, you know, this much topline revenue. And so, you know, you end up having debates and discussions and it's like very very draining. So, you said you were pulled into Google X and you ended up taking the interview and and doing well. I'm curious, what was it like entering, you know, Google or this like fang style big tech? And were there any cultural differences that stood out to you? You know, few than you would think. I would say the biggest difference that I saw there was there were really really really smart people like semantic had some smart people but again it didn't have an engineering culture even when I left in 2016 it was starting to develop one but it was really you know it was more a little looser gooseier than a Google for sure um but the quality of the people in Google X and X were really very high quality in terms of intelligence now what what seemed about the same was that many people in X as there were many people in Semantic didn't have good taste, research taste if or or project taste. And so a lot of people were really smart, but it wasn't clear that they were picking projects that would be that would land or you know or that were feasible in you know at least in my opinion. So I think that's a that is an attribute of engineers no matter what company, no matter how intelligent um people are. Um, but it was uh, you know, like it was it was startling how much how much brilliance there was. And I do remember like there was one guy who was clearly like over a 200 IQ. The guy was just you talked to him and he was just astoundingly brilliant and he was still in L4. Yeah. Why was he in the L4? Because, you know, he had lack of communication skills. You know, worked on really interesting stuff that was interesting to him but not necessarily had business impact. Didn't collaborate well apparently. you know, like there were things, whatever it was. And it didn't matter that he was brilliant. Like he was twice as smart as I was, but you know, just because you have intelligence doesn't mean you're going to be successful. And so that was, you know, saw the same thing there. If I'm understanding correctly, if you're very ambitious and you really want career growth, intelligence is not that important. It sounds like there are some things that are much more important. You cited uh communication, soft skills, project taste, picking things that actually matter. Yeah. Is there anything else that you that comes to mind? There are definitely people who are less intelligent. You're not going to like at Google we didn't have too many of those people like there were people that were really really you know most people were really quite smart. So I would say a baseline is you need to have a baseline level intelligence but I for instance don't think I'm a really intelligent person. I I take a forever to learn new things. I have like this ramp which is like this you know at least internally that's how I feel. You don't need that much intelligence be successful but enough. But um so communication skills um collaboration skills are really important like knowing how to work with somebody and not just piss them off because you're saying you're wrong but figuring out how to you know you know how to how to give them what they need in order to get what you want which by the way I haven't really mastered yet. I've screwed that up a bunch of times too. But that's one like we talked about business outcomes. I'll generalize that. I would think something that's really important to move up is focusing on outcomes. So this is like a really important thing and I probably did some of it subconsciously or unconsciously and some of it after I learned about it more consciously. It's very easy for people to focus on their own outcomes. In other words, they know what they'd like to do. They know about the technology they want to build. They know that they want to make it 10% faster or whatever it is or but often the outcomes of the company or the outcomes that somebody else is trying to meet are different than your outcomes. And if you don't project yourself into their shoes or the company's shoes and identify what the company's outcomes or divisional outcomes are or the other team's outcomes are, people are not going to be interested in what you have to do, even if it's really complex and hard and interesting for you. And so I think what really helps far more than intelligence is focusing on what outcomes need to be solved for a project or for the company. What metrics matter for those outcome? Because often there are things that don't really matter that much like you know there are like requirements that are unimportant and requirements that are super important. So focusing on the most important requirements and doing that work and focusing on the most important outcomes for your division or company and then focusing on the only the most important requirement is going to get you far much farther than being intelligent. I would say I I think you you have an interesting perspective because you're in academia because you're lecturing at UCLA, but you've also had uh a lot of success um in the in the industry as well. When I was in school, everything was very obviously uh intelligence-based. Maybe unless there's a uh well, aside from hard work, but you know, you're there's a test. You either get it right or or not. aside from group projects and things like that obviously coming from that place you know intelligence feels like everything and then you get to industry and I agree with you 100% intelligence is not everything in industry but because in in college it's very obviously meritocratic whereas in industry there's other things too like do people like you or uh you know other things like that would you say that career growth is meritocratic in the industry you In my experience, it was the there were cases where people were being promoted because a vice president basically pushed really hard or SVP pushed really hard and said, you know, they need to be promoted, period. Otherwise, we're not going to be able to keep them. And that's never a good reason to promote somebody, right? Because you don't uphold standards that for everybody else to look at and then you end up with a bunch of people that are not great and everybody's like, well, why shouldn't I be promoted because that clown is promoted, right? Um but by and large I would say it was meritocratic. You know I remember when I was on these committees we would look at the accomplishments and the complexity of the accomplishments the impact that they made uh for the company. We look at the communication skills. We looked at for instance patent portfolios. Were they helping the company with intellectual property um which was important back then I don't know how but important it is now but um so it was generally pretty fair and I saw Google too. It was very fair. there were, you know, very reasoned discussions about each person. So, I think it is. I think it is. It's just it's not just, you know, you ever see those charts where they have like a circle and then they have like little like a sort of a polygon inside, right? Right. And it shows like how your intelligence is versus how your your technical work is and it had to be pretty, you know, pretty wellrounded for the senior levels or or at least be really good in some areas. So then at Google, you went to Google X working uh in a new cyber security division and then a company was spun out of that, right? Chronicle if I recall correctly, but it's still under the alphabet umbrella of companies. That's right. Which was then reacquired by Google Cloud. Could you share a little bit more about that story? Yeah, sure. So we started as a stealth project. Nobody knew we were doing cyber security initially. The project name was Project Lantern and this is inside of Axe. Um, and we literally started from zero. We didn't know what we wanted to build. There were lots of debates and we knew generally what we wanted to do. But we spent, I think, a good six months trying to just figure out what we were going to build. We then basically converged on an idea. We started hiring a bigger team, started working on, you know, building prototypes of the product out. Finally got to an MVP, started, you know, shipping, working with partners, um, which was great. You know, actually seeing real customers use it was super useful. and seeing their we would we would actually go into customer sites before we had the product and just watch how they did their work and saw where they struggled which was super useful. Um really interesting actually like watching cyber security teams work. Some of the people were like stoned you know they're clearly out of it you know like these are the people that are using your product you know for better or worse. So the product was a product that cyber security engineers would use. Yeah. So best way to think about in a nutshell about Chronicle's product which was called backstory was basically cyber security today is a big data game. Okay. And what you what you want to do, especially if you're trying to discover attacks in your environment or investigate attacks, which is a big part of cyber security, some of it's proactive, some of it's blocking the attacks before they come in, but a lot of it is they're going to get in and we have to know where they are, when they got in, what assets they access, and so on. And so, as it turns out, virtually all software and hardware that's used in corporations today generates huge amounts of logs. A firewall will have every connection, the source IP, the target IP, what protocol. Web proxies will tell you what websites were visited, uh what what again what machine visited them. You have telemetry like DHCP that tells you like what machine what machine IP is associated with the MAC address, associated with the machine name. You have email logs. You have client logs, what software was installed, right? All that data is super valuable for identifying attacks. Okay? But there's such high volume of that data that people couldn't really process it. And so the customers that we were starting to work with would use a competing product which I won't name but they would literally go they would ingest a certain fraction of that data very little of it because it was so expensive to maintain it and they would go for coffee for 30 minutes while waiting for a query to finish to look up just one piece of information in that data. And so we said, look, we have Google planet sized uh compute, you know, and storage. How could we totally turn this around? And what we did was we built a product that would ingest all that data, pabytes of data, like literally some for some companies, a you know, pabyte a week or a pabyte a month, a huge amount of data of, you know, every device, every connection, every file installation, every settings change, like all that kind of stuff. And then we indexed it. That's what I worked on. That was my sort of you know addition like so that it would be more like uh at the speed of a Google search than a 30 minute let's go get some coffee. What would be a use case? Let's imagine that you discovered a piece of malware on a computer. Uh you might have a hash for that malware. You might know the IP address where it came down from or was downloaded. You might have the file name. You might know the directory was installed. Our product would allow you to take any of those artifacts and plug it in and then we would instantly tell you which devices also had that artifact on them, what related artifacts there were to that, you know. Um, so you could say, "Oh, well, this file had a different name even though it had the same hash, let's say." And so we know better check for this name because this might be on some other computer. So you could pivot. Um, we could tell you how many devices were impacted in your environment, who's those devices were, when was the first infiltration, when's the last infiltration, is it still active, and do that in like 2 seconds, you know, that kind of those kind of use cases. Wow. As opposed to 30 minutes where, you know, um, hopefully it gives you an idea. You know, when it comes to Chronicle, even just doing the research, I got a little confused. Sounds like it it spun out and then it came back in. What's What is the the benefit of doing that stuff? Why not just do it, you know, within Google? And that's kind of that. That's a great question. So, I think X's initial goal was to create viable businesses, you know, that could spin out and, you know, be world changing. Okay. And so, we were following that playbook, which was, hey, let's incubate it. Um, let's get really good people to work on it. We had really smart people. um let's not en encumber the team as much as we might a normal Google team. Let them work fast. Let them do what they need to do and then let's spit it out. You know, I can't really talk about why they required it. Partly because I only have hypothesis I don't know, but let's just say that it was a tight fit with Google Cloud. In other words, Google Cloud offers services to customers. This was a cloud hosted service. It used a lot of storage and a lot of compute which by the way Google cloud had and Google cloud could build for right and so I think that for various reasons which I can't talk about after we spotted out we were still an alphabet company like Whimo okay so we were still like a under the alphabet umbrella we were the C in alphabet for Chronicle um but I think they thought you know they had competing products they wanted to integrate them there were a bunch of reasons that they brought it back in and sort of integrated it so we were for a time not part of Google. You know, for instance, Google, I don't if you ever heard, but Google performance are notoriously lengthy and timeconuming and you have to write pages of stuff about yourself and what you did and PRs that you did and all the all the stuff, right? At Chronicle, we're like, we don't want to waste time on that. We want to build a project. So, as soon as we spun out, we had one page performance reviews and I think they were in a Google slide. It wasn't even like a big page of written stuff. So, so we could move more quickly, you know, and so that was great while it lasted. Why is it that you eventually left Google? That's another one which has to do in part with imposttor syndrome is a regular thing in my career. And in part it has to do with wanting to work at a startup as opposed to in a a bigger stodgier sort of Google environment whereas things are more slower moving and there's more regulations and things you have to deal with. Not that we didn't have to deal with those things in Chronicle, but more so. Um, part of me didn't feel like like there were interesting problems that I could have done. For instance, the storage architecture that we came up with, part of which I built and I think my code's still in there. I feel very proud of that. Um, you know, six, seven years later, whatever it is. Part of that was an architecture which was very expensive and there were ways to basically move that into different file formats and get off of things like Spanner which was you know very heavy weight where I could have dived in and tried to solve those problems and they'd be nice big juicy problems. um I didn't have the confidence in myself to do that and I you know I was afraid especially when you get to senior levels there are very high expectations for you right and so like if you go off and try to do something and then people are like well what have you been doing the last couple months and it didn't land and like you're like oh I tried you know I didn't feel the confidence in our leadership that I could go off and take those risks and have them have my back at semantic I did like I knew my bosses for years and so they they just knew that if I were going to go do something I'd either be successful or if I failed it was for a good reason and they'd give me the rope. Okay. But at Google, I didn't quite feel like I had that. And they offered me other roles, too. They said, "Oh, you want to work on secure databases." Um, there were some really interesting things there of like how do you compute and do database queries entirely in encrypted space rather than decrypting and doing it, you know, basically taking private data and exposing it where malware or other attacks get to it or interesting things. I really wanted to try something new sort of over cyber security and while I was at Axe I was talking to all the Whimo guys especially before Whimo even became Whimo and I was learning all about self-driving cars and so that was what I really wanted to do and that's why I decided to leave. Yeah. Before we get into the self-driving cars, I'm curious cuz I talked to someone who felt uh the the high expectations of the highest levels was a little bit limiting or kind of put too much pressure on them and they requested a demotion. Did you ever consider something like that? Not semantic. At semantic, I could do whatever I wanted to and it didn't really matter like you know. Yeah. But but at Google I had thought about those types of things actually you absolutely came to mind. The problem is that even at lower levels, there's certain expectations that I probably wouldn't have met if I want to go off for a couple months and just think about something and which is the way I've been most successful in my career. And so, you know, I got to be honest with you, um I'm a loosey goosey kind of guy. like I can produce prototypes and and optimize algorithms and do you know relative interesting stuff but when it comes to dotting every eye crossing every tea making sure I test every edge condition in my unit tests that's not my thing I don't enjoy that but at a Google that's just like that's what you do and so for me that wouldn't that wouldn't have necessarily made a difference because I would have had to do the things I didn't like to do in order to do the things that I wanted to do okay going into autonomous vehicles so you went to it sounds like that was mostly because of personal interest in the space. Can you talk about how you were hired and the story behind that? Sure. So, at Lyft, um I actually had a former student from UCLA that was working at Lyft and he said, "Oh, you should come work here. It's really interesting." And I thought, "We'll never hire me because I actually tried to apply for Whimo when I was in Google in X." And you know, again, this is another problem with being very senior. when you're very senior and you don't have domain expertise in a new space, they're less likely to say, "Oh, we'll take a we'll we'll try you because you're very expensive, right?" And, you know, you might not have the skills that they need, and you know, they're paying somebody that they they can't use. So, Whimo didn't want me inside of Google or Alphabet. So, I said, "Well, they're probably not going to want me because I don't have any experience here, but why not?" And so, uh, my former student arranged a meeting with their president. We had had uh coffee and then I went through a set of interviews again 78 interviews. Those did have some coding um and design problems. Um that went fine. Fortunately, there was no dynamic programming because I can't do that. I mean just can't like you know maybe certain cases I can but you know I'll fail every dynamic programming interview uh question. I was hired. What were the projects like or what was the thing you were most interested in working on that you did? So for me the big project that I sort of was proud of was transforming the architecture um from one which was a classic robotics architecture like you would have seen in the you know even in Whimo until probably the mid mid little middle 2010s which was largely handwritten algorithms and you know sort of not expert systems but handwritten algorithms that would make decisions like oh it's time to do a left turn let's run the left turn decision decision making system and figure out is it safe? Do we initiate? Do we not initiate? Um those systems you know in the mid210s were using um neural networks but they were only using it for vision. So in other words you know you know recognizing vehicles pedestrians and so on maybe their you know their you know angle and so on and their and their velocity and acceleration. But at Lyft they were using that sort of earlier approach which I'm sure came up through places like Carnegi Melon where a lot of it was handcoded and to me I looked at that especially during my time at X seeing neural networks and semantic even seeing neural networks and said there's got to be a better way because if you start hard coding an algorithm to figure out how to do a lane change and then somebody swerves in front of you now you're doing an avoidance maneuver right now avoidance maneuver do you switch out of your lane change algorithm in order to do an avoid avoidance algorithm or do you stay in lane chain and handle avoidance it just made no sense and so you know uh the team was also h the stack was also hand parameterized so literally you know we they're twe how diff how close do we want to get to the curve you know um okay how close are we willing to get to pedestrians what about bicyclists what about you know what okay so if we get too close to a bic bicyclist today let's tweak that and make that bigger but wait a second now it's gonna affect our our curb distance and so like there you know, a hundred parameters that all had to be tweaked and it was a really difficult problem. And by the way, companies like Tesla were doing this until recently too that that approach and Whimo was doing that um for a long time, is a dead end. Uh it's a dead end. And so the approach that you know I think that probably Whimo is using now I don't know but my guess is uh and I know that Tesla is now using they've announced it is you know an endto-end uh neural network-based uh perception and behavior planner system and prediction. you know, basically all them, you know, there might be multiple heads on these neural networks and there multiple functions that they're they're performing, but effectively it's a single or small number of networks working together to figure out the the movement. And so figuring out doing and doing a lane change is not necessarily a lane change algorithm, although there may be a little bit of that, but it's mostly about, you know, we know we need to go there. We know there's a left turn lane. Let's, you know, let's start maneuvering into the left turn lane and turning on the signal. And so my pro the project that I was most proud of there was basically designing with the head roboticists of the team were who are old school by the way an architecture which would accommodate basically an endto-end neural networkbased driver but put guard rails on it. In other words have a safety layer that would ensure that if the network went off the rails and told it to go through a red light we would slam the brakes. The safety layer would take precedence over the the system. But the safety layer was there only for basically ensuring you know that collisions never happened. Basically bringing you to a safe shop or basically you know following legal rules that the neural network may not do perfectly. Does that make sense? Right. That makes sense. And so that was that was I was very proud of that project. Unfortunately this is a this is like a bit painful for me. I didn't get as much credit for that as I I would have liked given the work I did which is like one of the things I learned is like you really have to toot your own horn. you have to really talk about what you're doing. Somebody else took credit for a lot of that. But the hard and really great part of that was this was no coding really. This was all about working with roboticists who did not want to change the approach and getting them to consider a new approach, working together to define the approach together rather than telling them how it should be because I didn't quite know. But by the way, I I thought I had a better idea, but they didn't want to hear that because I had no degree in robotics. and then eventually coming up with a product that was a collaboration where they were able to buy in and push it themselves if that makes sense. Right. Right. And that was all like, you know, influence and not because I and influence but not based on my skill because I don't they didn't have any autonomous vehicle skill. I think that's a common topic that people wonder when they're doing shared projects is how do you make sure you get credit for what you worked on? And so maybe you could talk about that. I wish I had a good recipe for that. I, you know, in general when you're when you're more junior, I think it really makes a lot of sense in the moment when you finish a project or you're you're almost done with it to take notes on what you did and what the big accomplishments were and what the metrics were so that when it comes time to write up a promo packet or even, you know, your sort of yearly review packet, you have all those details which you're going to forget later on even like two years later when you're going up for more senior promo, right? So having that I think is useful. To be honest with you, I never did that. But like in retrospect that wouldn't have been very useful for me because I would come out of it from my head and then ask people like what was that? How much faster was that? Like what what now was able to detect that we could have detected before? But I think that does help a lot and you have to toot your own horn like you know when you're when you're writing your performance review without embellishing. I think embellishment is really bad actually. But, you know, really state what you did and the value added and the sections that you worked on. Don't claim credit for everything. Claim credit for the parts that you worked on because I guarantee you when a committee is reviewing your packet, they're going to be like, "Wait, why are they taking credit for all this when we know that so and so did all of this great work?" And then you lose credibility. So, um, because I've been on those committees, right? I've seen that. I've seen that happen. So, you know, be very specific and granular about what you did, what the benefits were, how you collaborated. Um, I think that helps a lot. When you're more senior, it's more difficult because it's it's a lot of soft power. It's a lot of, you know, influence. I was actually reluctant like I didn't even try to take credit because I didn't want to alienate my co- collaborators who also were part of this. And so, I didn't go around and start teles talking to the president saying, "We've come up with a new approach." I let them talk about it. I let their boss talk about it and that actually hurt me because I never got, you know, when it came to review time, they said, "Oh, their manager initiated this project." I'm like, "What?" Like, really? This is news to me. Oh, no. So, yeah. Yeah. So, that was very I have PTSD from that experience, I have to say. Uh, before we leave Lyft, I'm kind of curious. Um, because that space is super competitive. there's like, I don't know, a billion different self-driving companies, especially back then. Um, what did it look like for Lyft to kind of win in that space? You know, it wasn't clear that we had a strategy to win in that space. We were, you know, definitely a nent organization. I think they'd been around a couple years when I first started versus like 10 years for for Google and, you know, X working on that technology and then Whimo. So, um, I didn't go in, for instance, thinking that I would be building the next generation of self-driving car that would actually overtake a Whimo. I I went in thinking this is an opportunity to learn something new and work with really smart people. Um, and that was my outcome that I was trying to achieve. Um, so I don't know if there was a plan necessarily. And the division was eventually sold to a Toyota uh subsidiary. So, they got the IP, which is great for them. And you know, at that point, I said, "Okay, I'm retiring now." You know, I'm kind of curious because how did you get into actually becoming a professor at UCLA and lecturing there? I know you get a lot of joy out of it. What's the story behind going back and and lecturing? So, back when I was in my late teens, early 20s, I was teaching programming in a place called Learning Tree, which you probably have never heard of, but Learning Tree is a for-profit school where they will teach you gardening, guitar, knitting, and back then they started teaching programming. And it wasn't very easy to find people who could teach programming because it was early. That was probably in uh 1990 1991. So I applied because one of my friends was doing it and I really enjoyed it and I was teaching people from Dvry. You ever heard of Dry? Oh, I've heard of that. They used those commercials, right? Well, they were trying to learn from me a learning tree so they could teach at DVY back then because they didn't even have a programming class there, right? Okay. So, I've been teaching people and really enjoyed it and it felt really good to uh uh get up in front of people and explain things and try to be really clear and uh so I had had some experience doing that. And at UCLA when I was in undergrad uh I would teach little classes. We found a room in the evening and I just invite people who had problems with some of the material and we'd just go over it on the whiteboard together or blackboard. Um so I enjoyed that kind of stuff. And when I was at Semantic, one of my colleagues was uh a guy who was a part-time lecturer at UCLA. We were having lunch and I said, "Oh, you know, I really enjoy teaching and he's like, "Oh, you should apply." And I'm like, "UCLA would never hire me. I don't have a PhD. It'll never happen." And I uh he said, "You know, let me bring you an application just in case." And I said, "Okay, whatever you want." And like a couple days later, he brings this application. He's like, "Here." Filled it out, gave it back to him. Didn't hear anything for months. I don't remember if it was 6 months, a year, I don't remember how long it was. And two weeks before fall quarter of 2001, so like December of 2000, I get this call, a frantic call from UCLA, can you still teach because our lecturer bailed on us? And I said, of course. So I had two weeks to plan a curriculum and basically teach undergrad course at UCLA. And you know, it's 25 years later now. So I'm very glad that they did end up picking you because you are one of the best lecturers and I think so a lot of my peers think so as well and that's why I kind of want to ask you how do you make these CS lectures so engaging and interesting. Well first of all it's very kind of you to say that. Um so there are a couple things that go through my mind. So first thing is remember I told you I didn't think I was that smart. So I think whatever intelligence I have and whatever that is whatever level that is helps me write better lectures because I feel like unless I could understand something myself being I think sort of slow other people can't understand it too and so I try to design lectures for what I think is one of the lower common denominators which is myself and I don't try to teach to the top 5% of the class maybe that's a problem for some people but I try to teach for the maybe the 30th percentile or 50th percentile and I think like what would what what what would I want to know if I were being taught this for the first time? I try to have empathy for the student. I think like where are they coming from? What have they learned about? Have they do they even know this concept? Should I introduce this first before I do that? So I think a lot about like I try to put myself in their shoes and ask what would they know? What concepts were they going to be fuzzy at versus concepts I'll pretty have pretty solid where I can just use the concept and explain it. And so that's a lot of what goes into my lectures. And so just for now, like I'm I'm trying to prove some slides. I'm literally spending days back and forth with chat GB03 discussing concepts and trying to simplify it so much but still get the essence right and then I'll go to Gemini and verify see figure out where they have differences and then I'll because there's a lot not a lot of materials on the internet that are actually really good to be honest with you. Um and the textbooks all suck too. I have to say we talked about outcomes earlier and for me an important outcome is that students not only learn something but enjoy the process. They don't I want them to have a good time and so I'm always thinking when I'm making slides, can I make them funny? Can I make some some joke or something silly? Can I make them like sort of colorful or you know add like an emoji or something to make it a little bit more fun so that they there's just a little bit of surprise when they come to class. They never know what they're going to see. might be a little inappropriate, you know, might be a little silly because I don't just want them to learn. I want them to learn and enjoy. They want to come to class. One one thing I'm also curious because I think a lot of people are scared of public speaking, but I you know, you're you're very good at it. Do you have any tips on on speaking well and practice just practice a lot? The more you do, the more fluent you're going to get. I even find like when I'm not teaching, because I'm part-time teaching right now, I'm retired. and I'm taking my dog for walks and working on side projects, playing with LLMs and stuff. Um, and I find that my speaking deteriorates over time when I'm not actively using it even over the course of a year. That might just because I'm getting older or whatever, but in in general, I found it too like earlier in my career. So, if you want to get better at presenting, if you want to get better at communicating, um, you have to practice a lot. And so that might mean getting a lunchroom and giving a talk about a project you're working on so that you can explain to other people what you're doing even if you don't need to. You're not doing it because it's required to transfer over your project or to integrate some technology just to do it. And people love that, by the way. And you'll probably screw it up the first couple times. You'll get better and better at it. And eventually you'll find that people will listen to you and take you more seriously if you're a great presenter, a great communicator. And in fact, story really quickly back to my time at Google X. Um, I remember I actually had a talk that I used to give at Semantic and I got permission. It was on stuck dad, I think, and on maybe malware detection. I had another one and I got permission from Semantic to give that talk in privately inside of X. And I remember people coming up to me afterwards and saying, "Wow, you know, you're one of the smartest people I've ever met." I'm like, "Little, you really don't know." But people will think you're intelligent and they will give you more credit and they'll introduce you to other opportunities based on your ability to communicate because people associate that with intelligence if that makes sense. Um and like I can go into endlessly about times in my career where communicating effectively helped my career. So it's super important. But yeah, practice you know with all the LLM and you know AI coming in are you seeing people cheat more often with LLMs? Are people learning less or more? uh last year in fall I allowed students to use LLMs not to write the whole project but for autocomplete or to write simple you know parts of the project which weren't really relevant to the material that we were covering and I think in retrospect you know that was a bad idea because I think people were autocompleting a lot more than just like a function uppercase a string or something you know like that kind of thing, you know, they were they were using it uh in ways that hindered learning. I've recently been reading a bunch of papers about how it helps in Hindu's learning and actually I don't know if you saw this recently a study at MIT that like synaptic connections are down from like 70% to 50% or something. Uh I forget the numbers but like I just saw a blurb when people use LM to solve a problem rather than working through it themselves. So I'm actually this coming fall I'm going to allow LLMs for learning like clarifying concepts asking for what what does this mean? But I will not allow them ideally for projects because I believe that it did hurt understanding and we saw that in exams like if you look at the exam understanding versus project scores there's a big delta perfect project scores but uh getting everything wrong. Although I have to say the projects were not solvable entirely with all those but you you know the latest models now you could probably get a 95% on on uh on them with really bad code but we weren't we evaluating correctness not code. How do you even tell if they're using LMS though? Um, well, one way you can tell is that when they autocomplete, often they'll autocomplete. These LMS will generate error checking, right? The error checking will typically have an exception with an error, a message, and the messages will be very consistent across different implementations. And so, in fact, we have cheating software that checks n squared different, you know, everybody's project against everyone else's project. And we will have flags where it's like 30% of this code is similar and a lot of it is like word for word similar error messages and similar variable names and you know like similar idioms and so like pretty obvious people are using these things extensively. You know a lot of people are worried about LLM's kind of automating software engineering and you know should they even get a software engineering degree anymore? You know what do you think about that? It's a great question and I and I think that it depends on what these models evolve into. Imagine you could literally give a project to an LM just like a junior engineer and it would produce a component which is largely correct with tests with security factored in with proper modularity DOI all the other good stuff you want. If we get to that point where bigger and bigger tasks of more complexity are solved correctly with good style and not no code smell and all the other good stuff that that is let's call that like AGI programming for for a time being for the time being. Contrast that with what we have today which is a models that can actually solve tightly specified sub problems pretty well build tests for them but you still need some you know some supervision you know did it use a good algorithm did it like do a deep copy when it should have done a shallow copy of a data structure like all this stuff like this right I think if we stay in a world where we get really good but not AGI good based on that definition I gave you earlier I think software engineers uh software you know software engineering will still be a great field to get into because someone is going to have to go and look at that code and understand the mission of the company and understand the standards and so on and then make sure that it's doing the right thing and that requires real thinking and introspection and corrections and you know and even if you have the model fix things you still have to know what to have fix um and so I do think that having that degree and having the skill of be able to write code and read code is super valuable and we could talk about if you want like but what about all the jobs going down right now you know if that which may be caused by the by LMS probably in part so that that's one situation the other situation if you truly have an AGI where you can go and delegate something to it and it will do like a L5 job it will do a really good job it might need a little bit of couple tweaks but they're minor tweaks takes on projects that would take weeks just gets them done um I think the world's a lot different and I and and in that world where literally all the software of whatever complexity can be written correctly securely with right tasks. Uh I think all bets are off and I think it's a different a different set of skills that are necessary personally and I can tell you what I think those are but I think it's different. What what are those skills? Project management soft skills those things. So in a world where software can be written like arbitrarily complex software can be written tested you know good style everything is you know good stuff like you'd expect of a senior engineer. To me, the world changes into a place where you're the where engineers are going to be focused and maybe they're not even engineers anymore on what we build, not how we build it. Okay? So, what problem are we solving? Why are we solving it? I think of in that world, the greatest engineers will be people who really understand a problem that they're trying to solve for a customer. That might be an internal customer, might be a customer that you're like a consumer, might be a business, and you know exactly what pains they're facing and how they measure success and what gets them really pissed and what is hard for them to do now and you want to make easy. And then figuring out how to really clearly communicate to an LLM those requirements to get it to do all that hard programming work that you would have had to do over weeks and months. And that is a really hard problem in its own right. So if people who have really great clarification skills, really great sort of uh outcome analysis skills like what outcomes is the customer trying to achieve, what are the metrics by which the customer measures success, what's important to them, what's not important to them in those outcomes, how can we communicate to a model in a way that tells a model what we need it to do and to meet those requirements because models will get some of it wrong. Those are the people are going to be successful. And in that world, I think you have big companies like Google and Meta and so on and Amazon. But you're going to have 10,000 smaller companies that are going to now be able to tackle problems like building software for pet sitters that never was, you know, tackled before, you know, or building software for like doggy daycare. I think about that because our dog goes to a doggy daycare and the software just sucks that they use, right? It's really bad. that if you have a bunch of domain experts who can use these tools, now you have a million small businesses each solving a problem in a way that's really perfect for those customers and not having to worry about the engineering. Does that make sense? So, it's a different world. Still a lot of software engineers, but different skills. What you said, it sounds like, and I don't know the product management function uh description too well, but sounds like it sounds like someone who's understanding the customer, the business, communicating well. It's almost like the LLM is like a software engineering team, but it's you know little query engine. It would be like a competent product manager and and I gotta say like in my life in my lifetime I've met very few competent product managers but yes you could call product manager would be a good name for it. Yeah. If program product managers were actually competent, most of them are not. I got to say what what makes a competent product manager by the way. You know, I think a competent product manager, a lot of them are good at communicating, although many are not, but a competent product manager in my mind is somebody who really understands again customer outcomes and customer metrics. So like for instance, if you've heard of jobs to be done or outcomedriven innovation, these are methodologies which are a more deterministic and and uh than just touchyfey. So they actually have meth methodologies where they say this is how you discover a customer's outcomes like what what jobs are they trying to get done during their day where are they struggling how do they measure success like what metrics are important to them like you know maybe you know like for instance when I'm brushing my teeth I drool I don't know about you do you drool when you brush your teeth uh sometimes yes yeah I drool a lot like I just like I got a lot of saliva okay yeah and for me like one of the really annoying things about brushing my teeth is it like like the drool's going all down my arm and I have to rinse my arm after I brush and it's like worn everywhere and like that's a that's a metric by which I judge whether my toothbrush is great. Of course, I haven't found a good one. Maybe I should design a toothbrush. But basically, um you really need to understand the pains that customers go through and what they care about and what's really not that important because a lot of things that you might think are important internally customer doesn't care about. And so I think most most product managers are touchy feely. They're like, "Well, I think I know what the customer wants, and I think I saw a really cool feature in this product here, so I'm going to go and do what they did, but do it a little bit better without really understanding what the customer is struggling with. Um, how often they struggle with that thing. Is it important to fix or maybe it's not like, you know, maybe it seems cool to you and maybe they feature because they have some extra team members that can do it, but that's not necessarily the right thing." And so yes, it would be a very competent product manager who really understands the customer, maybe worked in that environment, knows the problems, has suffered through the problems, and can then basically tell LLM how to solve that problem. I see. So, so competence, if I'm understanding correctly, is user empathy. It's knowing what actually matters uh for them and um building for that and communicating for that and measuring that, all those things. But a lot of people think that's a touchyfey skill like it's something you just sort of develop and you sort of have intuition about the customer. I think it is a a repeatable process done through interviews done through observing the customer. It's not something that you just sort of get better at by feeling it. It can be repeated is what I'm saying. And very few product managers will do that. I see. Most of them just say, "Oh, I know this product. I've been working on it for five years. I know the customer. I talked to Joe last week at you know customer name and you know this is what they really need." Do you really know that? Like you think you know that, but what are they trying to get accomplished? Right. Right. Product managers typically think in terms of features, not in terms of customer pain points and what they're trying to get done. I see. In my in my experience, I'm sure there are obstructions. Yeah. Yeah. Yeah. I'm sure there are. I just never met any. Yeah. Okay. Um, you know, coming to the end of the interview, I always love reflecting back on the careers and you've had a full career at this point, more retired at this point. Um, so I'm curious like looking back on your career, is there anything that you you regret or something that you wish you would have changed that maybe others can learn from? Yeah, I mean a bunch of things I would say like first of all, I should have left my job at Semantic earlier. Um, I feel like you should stay in a job as long as you are learning new things and building new skills and as long as you feel empowered to grow and try things that might be uncomfortable for you and not have to worry about your, you know, what if I fail occasionally, like have the space to try things and get them wrong, but to be to be able to create great things. And I had that during a lot of my time at Sevantech. But there was a point where I just wasn't growing anymore and I was stagnant and I didn't want to leave. not because, you know, it was interesting, but because I just didn't have the confidence to go somewhere else. And so, I would say there's a lot of value in staying in a job as long as you're growing. And that means maybe switching teams and staying in the same company. There's a lot of value of having that institutional knowledge of a platform that you're working on or set of systems of your reputation. Reputation goes a long way. If you're really good in one in one area, you switch to another team, you can use that reputation to help you in the other area often. Um, and so I think staying in a job for five years, even 10 years can be, as long as you're learning, can be really great. I don't advise people to switch every couple years necessarily. On the other hand, you know, staying staying too long, you know, you can get stale. Um, it gets easy to just say, you know, I'm comfortable. I'm not really having to be challenged. I can do my job. I can wake up and have some nice coffee in the morning and I do whatever I do. And it doesn't really, you know, when you get more senior, often you can just do that. And so when you get to that point, it's time to leave and and challenge yourself some more. And the problem with that is when you do leave, you do start over. Like I found that I had a pretty great career at Semantic. If you asked anybody at Semantic from the 21 years I was there who I was, people would know me. They'd say hi to me in the hallway. I even had people come up to me on the street because I would be on TV talking about Stuckset and stuff like, you know, I was on Fox and, you know, MSNBC and Wall Street Journal, New York Times and all that stuff. and it was really exciting. But then you go to Google and they're like, "What have you done for us? We don't care that you did those things. It only matters what you've done for us." And so that requires a lot of rebuilding up trust and building up, you know, sort of a reputation and doing a lot of good work. And, you know, that's stressful and it takes a lot more work than staying where you are. So, if you do that, got to choose wisely. I hear I'll probably misattribute who said it, but there's some quote about you should either be learning or you should be earning in your career. Um, what are your thoughts on your golden handcuffs somewhere you're no longer learning but you're earning a ton? Do you would you still advise that that person leaves? Um, I would never say don't I would never say leave if there's a a good package and you know we have to optimize for multiple things in our life, right? Learning is definitely important, but also being financially stable and different people need different amounts of money to feel comfortable about being safe in their life and having enough to take care of their family or themselves. I think there's a place for both and it can be okay to be comfortable and making a lot of money for a while, but I think that if you're solving for enjoyment and fun and hard problem solving, which a lot of people are, that can be toxic over too much time. Definitely. It sounds like early in your career you were lucky to find what you enjoyed. A lot of those early problems were super interesting. Yeah. Do you have any advice for people who um want to find what they enjoy in software engineering? Yeah, for people who are in college now, graduating soon. Um I would say like try lots of internships if you can and that might be difficult now given the way things are going with jobs right now. Everything's cyclical though, but right now jobs are tough. Um, I found cyber security entirely randomly. You know, it wasn't something I was like, I want to be in cyber security. It's like I had an internship. My first year was doing file management. My second year was doing blah blah blah. Third year, oh, viruses. The more you can explore, the more you're going to potentially discover a passion. And like I think there are many jobs where you would think it's going to be totally boring, but if you go and start working on the problem, you're going to realize there's really interesting problems to solve. And you might find a passion for a field that you would never expect that you would enjoy. And then you're like, "Wow, this is really interesting. There's really fun problems. I like the people I'm working with. Customers are interesting." Or maybe they're a pain in the ass, but that's interesting to deal with. Like I would say just try things. Don't try to wait and find the perfect job. Find a job that where you have a good manager. Um there's some hard problems to solve. Um and um you don't know anything about it and you'll learn. Like that's you might find your dream job and that lesson be 25 years. So before you know what you enjoy, you're saying, you know, search with breath and then once you find it, then you can kind of go deeper. That's right. I mean, look, you might get lucky. If your first year internship is really interesting and and you're doing well, I would stick with that. You could try breath there, but I would say like a greedy breath. The last question I have for you is if you were going to go to yourself right when you were graduating from UCLA and give yourself some advice knowing what you know now, what would you say? It wouldn't be one thing. I mean, the biggest thing would be don't let fear of failure hold you back. You probably can do more than you think you can do. I might have had a richer career had I listened to that. I would say focus on the outcome. So whenever you're working on a project, think about who's going to use it, what do they care about, what how do they measure success, and then just optimize for that. And don't like try to make the perfect thing or try don't try to make a very, you know, round thing if all you need is a little square piece here. Like often problems can be solved um without having to be perfect and still be really good. So like focusing on people's outcomes. Big one is when you're when you're presenting, for example, like this had to learn. I used to go into presentations and talk about the technology to senior leaders and like in retrospect those senior leaders don't care how the algorithm works. They don't care. They just want to know is it going to be faster? How much faster? Is it going to generate more revenue? How much revenue? Is it going to, you know, fix a problem that currently takes 20 people to do it? Make it 10 people to do it. And so you really need to get in people's heads and think about what they're solving for. Again, this whole idea of outcomes and then speak to their needs, not your own. So that's a big one. I said find a good manager. I'll just say that again. A manager can make or break your career and your life and your like happiness. So, you know, a good manager that trusts you and gives you some rope is uh super valuable. Learn how to collaborate with people. Just like don't assume that you're right and just tell people they're wrong. You have to really learn to work with people and understand their point of view. I still struggle at that, but you know, it's something that's a work in progress. Those would be big things. Yeah. Yeah. Well, thanks so much, Carrie, for your time. I really appreciate it. I was really looking forward to it. I think there's a lot of stuff in here that people are going to benefit from. So, thanks so much for your time. That was my pleasure. Hey, thanks for watching the show. I don't sell anything or do sponsorships, but if you want to support, you can subscribe on YouTube or you can leave a review on Spotify. And I'm always looking for new guests to interview. So, if anyone comes up who you think you really want to hear their career story, uh, let me know and I'll try to reach out to them and get them on the show. Thanks for listening as always and I'll see you next time.