SPEAKER_00 0:00

And we are live.

unknown 0:01

Alright.

SPEAKER_00 0:02

Alright, this is awesome. Okay, well, hello everybody. My name is Valentino Stoll, and I'm joined by, you know, my co-host here, Joe Leo.

SPEAKER_01 0:11

Hey everyone.

SPEAKER_00 0:13

And uh we figured it was high time for somebody to just start talking about Ruby and AI.

unknown 0:20

Yeah.

SPEAKER_00 0:21

I feel like we have like a lot. Yeah, right? You know, there's all this Python out there, and lots of people are working with AI with Ruby. And you know, lots of people are are writing about it. But I feel like it could use like a little boost from the community to help highlight people, right, that are working on the stuff and exposing it. We have the lovely uh you know Ruby AI meetup in New York City. That's uh what is it? What is it called now? Augman? Artificial Ruby. Artificial Ruby. Yeah. But it's a real meetup. It's a real meetup. Yeah, it's uh it's pretty much like a conference at this point, I think. Yeah, no, it's huge.

SPEAKER_01 0:55

It is uh it is like one of the more life-affirming, career-affirming things that have happened to me in the last five years is showing up to that uh that meetup every month. It's so fun.

SPEAKER_00 1:05

Yeah, I agree. I mean, that's where we met. Yeah. You know, just fantastic. It's just super I mean, open AI was at the last one, like talking about their Ruby gem. Right.

unknown 1:16

Right.

SPEAKER_00 1:16

Right, like it's like really getting like official, I think. Yeah, I think so too.

SPEAKER_01 1:21

In that, like, yeah, you're right. Like, OpenAI coming and giving that talk to me from like, okay, this is actually where you go when you're releasing stuff in Ruby. Yeah, that was that was a big deal.

SPEAKER_00 1:35

So, you know, I'm I'm AI team at Gusto. You've built this incredible AI company, Phoenix. Let's let's just hash out like you know what what we're working on and set the stage here. Uh, because I would love to dive in kind of more to Phoenix and like how how did you even what is it and like you know, dive in.

SPEAKER_01 1:52

Well, we can talk about Phoenix this time. I think next episode, you should just tell us all of the trade secrets going on at Gusto now that you've been there for a few weeks. You know, we'll just ask an answer and you'll just you know violate the terms of your of your employee agreement and tell us all about it.

SPEAKER_00 2:08

Uh I mean there are some things I could talk about. Uh I'm happy. I'm happy to talk about it. Yeah.

SPEAKER_01 2:12

Yeah, so um, yeah, so let's talk Phoenix. This is, I think, uh an interesting place to start. First, because I know all about it, and I think second, because it is something that was built, is built and being built every day to serve the Ruby and Ruby on Rails communities. Um, and I don't see a lot of that. I mean, and I and I understand why, like from a business perspective, you think that the surface area of your of your business is going to be much higher if you either serve all languages, like a lot of the um AI rappers out there, or if you tailor your things to JavaScript, which we've seen, or Java, which I've seen on the enterprise. But, you know, this is something that I I hear at Artificial Ruby and that I I deeply believe. The creative things that are gonna come to define the next generation of software development, some, if not most of those are gonna come from the Ruby community. And uh, in order to do that, we've got to give them some good tools. And uh, on top of that, it's just way more fun. It's just way more fun to be doing this in Ruby.

SPEAKER_00 3:19

Yeah, I agree. You know, I have definitely had my fair share of Python. And you know, to be honest, it you know, the tooling is there. You could just run stuff and uh I wouldn't say not worry about it, but it is easier, more ergonomic. You know, and it's getting better in Ruby. You know, the sport is coming, so I don't want to put people off saying, you know, saying go use Python. I think we're like kind of at that tipping point where there's enough in Ruby to get the ergonomic feel of a Rubyist style of AI system.

SPEAKER_01 3:49

Yeah.

SPEAKER_00 3:50

And I I feel like, you know, now Rubyists are great at you know building the everything to suit them, right? Like Yeah. I'll never forget, uh I forget on the Ruby Rogues we had somebody where, you know, they had built their own, like, you know, their own like high performance throughput HTTP server. And I was like, well, why would why would you build this with all the other options out there? And it was just like, well, I could.

SPEAKER_02 4:14

Right.

SPEAKER_00 4:14

And it's like I know how it works, and it works great for my use case. And uh, you know, I feel like that's like almost the Rubyest way. It's like you know, like you get like these, like, you know, all the meta programming, and you can like create like kind of your own language out of Ruby, and that's like the benefit of Ruby, right? Is like if you want to make your own DSL, like honestly, it it's very easy. Make it look how you want it to read, and it's honestly that easy.

SPEAKER_01 4:42

Yeah, yeah, I totally agree. And I think that's so that is part of our journey in developing Phoenix. We started last year or sometime with Q3, and um, you know, my development team, they're not they're not all Rubyists, you know, they're they're kind of generalist engineers. Um I run a company that does a lot of services, uh software engineering services, teams gotta be able to hack in in multiple languages. And so, okay, so they're looking at the tool set and saying, yeah, it's mostly Python. Like if you wanna, you know, if you want to start to integrate with the APIs, like the LLM APIs and language at the time that we wanted to work with it. You needed to use Python, or the other option was like, okay, well, there are these ports from Python tools to to Ruby tools, which I think is you know a similar story to the the HTTP server, right? And it's like, okay, well, that's good, but I also know, you know, we've been in this Ruby community for a long time. Like some people do it just has to be Ruby. And so we want to know, like, okay, well, is it is it a tool that's that's actually like serving a purpose and you know and well maintained, or is it just a guy alone somewhere saying, you know what, I I refuse to code in Python, so I will rewrite everything in Ruby before I even do anything. And uh so, you know, so in the beginning we did a lot of Python, and uh today the application is much more Ruby and Ruby on Rails. And what we learned, you know, courtesy of Scott Werner, among uh you know a number of other people, is that okay, now there are starting to be some some tools that are built from scratch from Ruby. They're not built, they're not porting anything, they're doing something net new, in that case, the Gentec architecting and orchestration, and uh and they've got a life of their own. And that, I think, as you've mentioned, is kind of a tipping point. It's like, oh, okay, this is this is where we're headed next.

SPEAKER_00 6:32

Right. Yeah, and I mean I I feel like at the last uh Ruby Kayegi, Matt's even said it best, like, you know, the the language of Ruby is like almost built for LLMs, right? Like the way that it reads is like uh so much like a sentence that like you know, you could almost just like write a Ruby program so that it is sentences. Yeah. Or maybe get like some more, you know, more advantages out of there, like uh, you know, markdown to Ruby. I think that's entirely possible.

SPEAKER_01 7:00

Yeah. Somebody mentioned uh that they did they thought that Ruby wasn't kind of curious about this, that Ruby was not well suited for I guess well suited for the kind of uh code generation that happens today with LLMs and a chatbot, which I've got some opinions on we can get into. But like that style of development today is uh like Ruby not a good candidate for that because it's dynamically interpreted, it's you know it's strongly typed, and this person's contention was like, look, TypeScript, Python, and uh, you know, and then the Java.net kind of suite are going to be the languages uh that continue to prosper because LLMs can easily spit out code based on them. I'm curious what your thoughts are on that.

SPEAKER_00 7:47

I mean there are ways around that. I I honestly agree. Currently, I have this project I'm like kind of working on the side to like fine-tune an LLM specifically for Ruby code generation because of this reason. And you know, it's it's challenging. Like, I will say. And the the biggest the biggest issue is like these bigger models, they're good, right? Like they're honestly good enough at Ruby uh today that like if using like I don't know, cursor or even anthropic code is pretty good. They can generate like if if you want to just spin up a Rails app and work on features, like Claude Code is great, like you know. Yeah, like expensive, right? I heard actually the the author of uh you know anthropic's uh Claud Code, uh they had him on Latent Space podcasts, and he was basically saying, you know, listen, like it may seem like you know, fifty dollars a day or a hundred dollars a day may be expensive, but like you have to think about like what it enables the developer to do, right? Like if developers are expensive on their own, and if you can enable them to get even like a fifty percent boost in productivity, like that's very valuable, right? Uh because their output is even more so than the these code generation aspects of things, right? So like it'll be interesting to see, like, you know, yes, it it's expensive, and so like the free models are honestly not great for Ruby code generation, right? And so like, yeah, if you wanted to run something on your laptop, right, like currently, yes, Ruby is not like ideal for code generation in the same ways, right? Yeah. But, you know, like it is the tipping point, right? Like, it's not like so bad that it's like preventative, right? Like you can still, even if you don't know Ruby, you can still use these free models, right, to learn Ruby. Like it'll still generate enough code, you know, enough quality too, where like it's not gonna send you down necessarily the wrong uh you know pathway, uh, although that is still an issue, right? Like I'm curious what your thoughts are there, like uh because I know like it's almost like I feel like it's like relearning how to use a computer, right? Like we're almost like relearning to like how to use a computer, how to program, like how to do all these things, because like there are ways of like kind of massaging the uh you know the chats or experience of building software with these new things that we're kind of still like readjusting to, right? And I know I've had I've had issues like you know, and maybe that I'm like a little bit out there because like I have a very specific I've read a lot of software books, so like my thinking process is around what I've read and what I've experienced and like you know all the things that I've made. And so I'm like biased, right, to like whatever it's generating. So like I have a harder time like assisting the path for the LLM to travel down rather than it just like taking its own path. So like there are like these kind of like two, I feel like buckets where like, well, do you want to just like produce something or do you want to produce something very specific? And I feel like right? If you want something very specific, it's like much harder to get that than just like making something happen, right?

SPEAKER_01 11:09

Yeah. I uh I I did uh I just did a talk a couple weeks ago in uh at an Earth conference, and it's uh and what I talked about was the cognitive burden placed on development teams because of generative AI. And uh and it's interesting that the um that they said on latent space, hey, you know, even if it's a 50% increase, that's really worth the the money. I don't disagree with that, but it's right in line with one of the stats that I pulled, which was a Microsoft study. Uh of course owns GitHub and can measure pull requests and all these different things. Um and they said that yeah, they're seeing a 55.8% increase in what they're calling productivity, which is really just code generation and pull requests and acceptance. Accenture did a separate study, which is interesting, and they were measuring uh code compilation. And so, okay, that leaves Ruby out, but for a lot of applications, it's like, okay, well, the amount of times that you're compiling code means that you are testing it more, you're releasing it for some definition of production. So, you know, what I was trying to try to hammer home in the talk was if you're if you are looking, like if every single day you simply have code in a wheat surf or a wheelserve clone or cursor, you have this amount of code, and you are reading it, messing with it, you're putting it into 50% more code than you've ever seen in your whole life, and then you make it every single day. And the decisions that come up from that are, I think, fascinating. Because just like you said, you've got a really specific idea of where you want the code to go. I do too. But that means that you and I are gonna do it a little bit slower than the next person. And I'm not sure, like maybe we're really confident, you seem like a confident guy, I think I am too, that hey, I'm doing it my way and it's gonna work. Right? But the next person next to me might be moving a little faster. And if I was a mid-level engineer, I might be like, well, I don't know. I mean, these people around me seem like they're going pretty quick, and I'm not going so quick. Maybe it's okay. You know, the same as doing what I expect, stand up locally, stand up and server working. Maybe it's okay to do this. Everybody has looked at a pull request they were supposed to review and immediately saw that it was AI generated. And every hand went up. And I'm I mean, me too, you know, and I work with some really smart people, but I can tell when I look at it, I'm like, oh, this was generated by AI. Right? Like some of this is you know straight from the code generated. And uh, and I I'm not saying a good or bad thing about that, right? Because if they're not a very developer, maybe you're happy that it's generated by AI.

SPEAKER_00 13:51

I'm curious there. Like, what what are your tells that it is code like generated from an LLM? Like, are there are there very specific examples, or is it like kind of just like a gut feeling?

SPEAKER_01 14:04

Well, if there's a bunch of comments in the code, you can tell right away. They're always trying to put in a bunch of comments. It doesn't I it always drove me nuts to begin with. Beyond that, I think what you end up seeing, I mean so it depends, right? Uh there can be really insane things if really the people don't know what they're doing. Like you see, like when when AI is sitting on top of a an application that already is not really making a lot of sense, it will do insane things. Like we saw once we're doing a consultant gig, we saw a commit come through that had a try-catch block where the catch was like rescuing from a certain error and then modifying the database record um to like to account for the fact that whatever it was looking for was not there. And so it's like a you know, like like generating a migration, right? Because uh, you know, it hit an exception. So there could be some crazy stuff. But generally speaking, I think, you know, there can be a there's a certain amount of verbosity, I think, and there's a certain amount of um you know editing in in sort of more files than maybe is necessary, and you start to see like, oh, I think that there's I think this is AI is at work here.

SPEAKER_00 15:22

Yeah. Yeah, it's funny you mentioned comments. It's kind of funny because like there's so many tokens wasted on comments, especially when it's like a descriptive comment for a very self-explanatory line. Yeah. Like just one, right? Like, oh, we're gonna update the blah blah blah table, and like it's like table like update.

SPEAKER_01 15:41

Yeah, it's out of date, yeah, right. Yeah, when that happens. And I'll call it out, you know, but um I'm always just like, you know, it's it's a weird thing. It's like, well, I'm gonna call this out and have this person change it, but I know they didn't really write it. Uh so it's not a comment about them, but it's kind of it's kind of more about their vigilance in accepting code and not in how they're writing it. Because I don't think they would write that. Right. Which is a kind of funny thing. And you know, so you made these different decisions that, as you've already said, you know, we're kind of still grappling with how that works on a day-to-day basis.

SPEAKER_00 16:15

Yeah, you know, it makes me think I'm thinking about your latest like Phoenix post on like reactive AI, right? Like where uh I feel like we're kind of at the cornerstone of like truly implementing uh you know AI actions as like a reactive component to various things in the day, right? And to the development process. Uh right now it's like very like you know, instructional where you're like directing it, doing trying to do something. And there are more and more things where it's like, oh, like GitHub actions that run on something, right?

SPEAKER_02 16:47

Yeah.

SPEAKER_00 16:48

But uh they're not like it's still kind of like, oh, it's gonna be a GitHub action and it'll create a pull request. And then you're like, you have to still have to, you know, get in front of it and like do something with that, right? And like, yeah, well, it's gonna produce way faster than you can review it, right? Like oh yeah, absolutely.

SPEAKER_01 17:03

In fact, that our our first customer, Phoenix's first customer was ex got exactly that. We're like, because we're thinking like everybody else was like, hey, you know, this thing can do a task, in this case, generate some tests and uh and open a pull request for their repository. Well, you're right, it can open a thousand pull requests, which nobody wants. But even if it doesn't, you know, it's kind of like this interloper that just gives you a thing that's like, oh hey, these tests work. It's like, well, yeah, but now I gotta stop what I'm doing and look at it. Uh and it's like having another member of the team that's like PandaBots, it's a little slower. You know, and it so it's like it didn't fail. The customer liked it, but we could see by how fast PRs were getting accepted. This was not the way. And I've come kind of full circle around that, and I like, you know, your term for reactive actions is is exactly right. First generation AI, the cursors, clients, bolts the lovables of the world. These are these are great tools. I mean, we use some of these tools. We use bolt to create Phoenix. I'm not saying anything bad about them, but I'm saying this is first generation. You have a chat, you have a chat with a thing, you tell it what to do, you give it some instructions, it does it you know, anywhere between halfway good and 80% good, and then you have it go back again. I think that that is I think those are not actually very long for the world. So I could be wrong. But what uh what we aimed to do was take that out. Phoenix doesn't have a chat interface, it already has its commands. Its commands are to like lovingly support your code base with tests, with bug reports, uh, you know, with suggestions or highlighting of risks, um, and to do this in an automated fashion, right? You know, it's gonna spin up immediately, generate a full suite of tests, and that's gonna follow you as your as your software evolves. And there's nothing stopping us from doing that, right? What it does take is a serious amount of automation and sort of well-crafted, well-designed um integration between a bunch of different endpoints. So that completely different.

SPEAKER_00 19:14

That's funny you mention uh dependabot, because that that's kind of like how how I imagine all these things to start, right? Like it's just like annoying you with like a barrage of like updates. Hey, this thing is like out of date. And it's like, well, like merge it then. Like, I don't know, like I don't have a test though, so like don't, right? Like you end up like trying to configure it just right, and you spend all this time doing that. Uh, and then you finally get it right, and it's like you still end up with a pull request that hangs, yeah. Because you still gotta test it, and like that takes time manually, right? Like verifying that things work and like some side problems.

SPEAKER_01 19:48

Like, I like I just got the we use Camal to uh deploy uh part of our application. I think Camal is cool. I just got a dependabut update, it's like, oh hey, it's a minor minor point release bug. I'm like, oh because I you know the test is manual, I have to actually deploy it a bunch of times and make sure it works. And I of course it's good software, but I'm like, oh, this is like another thing in my day.

SPEAKER_00 20:11

I will say I I have a little side project called now reading.dev. Uh it keeps track of my reading list so I could just throw stuff at it. And like I use Camal 2, and it was I was like, oh, bundle update, you know, minor, right? Like that should be safe. And I go and try and Camal deploy, and it's like, oh, you know, Postgres fails because you can't like restart update now or something like that. Like it was like a small like configuration option that like no longer is there in a minor version update, right? And like, oh if I had this autopilot, like I just couldn't deploy then. Yeah, yeah, yeah.

SPEAKER_01 20:44

Like right.

SPEAKER_00 20:45

Yeah, we're all just like deploy is automated, like it was just it would just be alerted, right?

SPEAKER_01 20:51

Like, yeah, right. Yeah, we're all just living in this, like, we live in this incredible amount of trust, like interdependent trust with semantic versioning. Where we could just say, okay, the minor version is gonna work. Because for most people it is, but it's not gonna work for you, and uh, and then you're screwed.

SPEAKER_00 21:06

Yeah, and that all this computer use stuff, like that seems a little more promising. At least it could like spot check things for you and be like, hey, like I tried to deploy this thing, and I see like, oh, your homepage doesn't load, right? Like, yeah. Yeah, that's uh you know, that'll get that'll that might get handy, but like I feel like it's still not quite there. I don't know.

SPEAKER_01 21:27

No, I think you're right. But you know, I think that it's coming.

SPEAKER_00 21:30

It's uh there's we just we're stuck in this place where we just want it now, and like it's it's not there, but people use it like it's now, right?

SPEAKER_01 21:37

Like, yeah, it's true. But I'm uh I like I have to say I'm a little like I'm a little disappointed in the software community writ large because I I feel like most of us are not thinking beyond what like lay people are thinking about AI. It was what three years ago now that ChatGBT kind of like over the world and open AI, the first thing they said was like, hey. Hey, this is just like one use case. This is not the thing. The chatbot is not the thing.

SPEAKER_02 22:04

Right.

SPEAKER_01 22:04

But it's three years later, we're all still treating it like, oh, well, if you've got a chatbot who will code you to a landing page, we gotta give you like $25 million. Um and uh and I think that's that's fine for the permit, you know. I think for us, you know, we should hold ourselves to a little bit of a higher standard.

SPEAKER_00 22:20

Yeah, I I don't disagree with you there. I've been trying to escape the chatbot as a input. Yeah. As like the definitive user experience. Because it's so easy, to be honest, if you have a software product, like a chat makes complete sense if you were just like trying to service somebody that wants something solved from you, right? Like yeah, just throw them in a chat and like chat ops, which is fun, which is funny, like chat ops didn't work before, but now that AI has you know conglomerated the idea of it, it's like now back and like everybody's like, oh yeah, chat ops. It's like oh, like what didn't work about it before? And you know, we're like relearning that aspect of things. But uh yeah, I'm definitely like I'm liking this more of like reactive approach of seeing like okay, where can you just like throw an LLM at just like observing something and like start there and see like what it observes? And like from from past experience, like it's both good and bad, right? Like it's like a whole new learning experience where like uh you know if you if you're trying to like categorize what it's doing, like it's honestly not that great at that. Like it can't it can be good, but it'll like it'll just make stuff up, and like if you have very specific things you're trying to like have it like bucket, it'll just like you know, it'll get it like 80% of the time. And then like, well, what do you do with the other 20% of the buckets that are just like you know, what is that?

SPEAKER_01 23:48

Yeah, right, yeah, and then you're going back through it manually.

SPEAKER_00 23:52

Right. But I I feel like there's more and more like observational stuff. So I'm curious, like, uh where do you see kind of like the observations heading to the point where and like where's the point where we can actually start acting on those observations, right? Like, how does somebody even know, like, okay, this is a better way than chat bot, right?

SPEAKER_01 24:12

Oh, I see what you're saying. I mean, what I see is you know, just stepping back to the the business perspective, like you have to be able to demonstrate a lot of value to people. And you know, the way that generative AI through chat demonstrates value is it spits out a lot of code. And in the case of Bolt, see Bolt is this fascinating example because Bolt is not really Bolt. Bolt is Stack Blitz. Stack Blitz has been around for 10 years, and Eric Simon built Stack Blitz to develop a tool that is production grade development environments in the browser, and they're working on it for 10 years and doing okay, they're selling it to the enterprise, but certainly not blowing anybody, uh, blowing anybody's doors in. And uh, they got this idea to wrap it around a chat bot. And the result was magical, not because they did something to a science and LLM, but because they already had this infrastructure that could make up, and that is a really cool experience. So people see that and they're like, that's the value I want. And I think that we're all just human beings at the end of the day, and we want to have that experience of like that thing. All of a sudden I have something that I didn't have before, right? And um, you know, for us, it's funny because we're we're these totally test-driven engineers, we write tests for everything, but we know that first of all, that's not the world. And so I know all there are limitations to what we can do. We can't possibly think of every end case. We can't possibly like test everything because that's why bugs creep into our system and we get things wrong. So what we want to be able to deliver is like I already I feel I feel the safety, right? Like within a week or week, or maybe even sooner than that, not only do I have these tools but this or these tests, but this this software has alerted me to issues that would have been in production and are now not. You know, I think that's that's something where we can say, oh, okay, okay, redesign belief I have the safety methods here.

SPEAKER_00 26:07

Yeah, I will say there's there's an incredible amount of value for what you've built at Phoenix. And I I say that because having personally tried to do TDD with all these varying code editing tools, um yeah, you know, like cur even cursor cloud code, like open AIs, they'll help, they'll get tests created, but it's like you start looking and it's like everything's mocked. Like, yeah, nothing's real, you know? And so, like, well, it's just testing mocks, like, yeah, I could do that. Like, I could get anything to pass, you know? Like, oh yeah, yeah, yeah.

SPEAKER_01 26:37

And so like it'll like hallucinate factories, it'll it'll do some really crazy stuff, yeah.

SPEAKER_00 26:42

Yeah, and you can have it and it'll it'll get on a loop and like you know, eventually fix itself, but like only to a certain extent. And uh sometimes it gets into a loop and you've just wasted hours of it trying to like recreate a database helper or like something like that, right? And like you're like, well, we already had one of those. Yeah, right. The context got too big and now like you've forgotten about like all the stuff that exists.

SPEAKER_01 27:05

Like well, that's you know, that's an interesting point with um you know, one of the reasons that that uh I think that the LLM stumble on uh on some of the tasks we give it is because the context wind, like they're they are they are loading on too much context. Friend of the show, Landon Gray, had a post about this not too long ago where he was like, if you're focused on the size of the context window, you're focused on the wrong thing. And I think he's absolutely right, because it is all about the specific context you give it. And it's still today the best use of AI when you're generating code, and I'm sure you've already experienced this, is to ask it to do very small, very specific things, or else it's going to do something it's gonna, or else it's gonna make a bunch of assumptions and do something you don't want. And so, you know, loading in the proper context is not easy. And even when I try to, I'm like, look at this file, then look at this file, and then do this, it doesn't do it. Um, you know, and that's using G41, it's using the latest and greatest, and it is better, but it's still not there.

SPEAKER_00 28:04

Right. Yeah, you know, I've been experimenting a little bit with sub-layer amongst other things, of trying to because like you make a good point of like, you know, the the context window being an issue, but also not being the right focus area, right? Like uh it's almost like we're trying to teach it how we want to do things and how we do things, and yeah. The source code isn't necessarily right the only source of that. It's also like our brains and like our utility and like how we do things right. And so like every person's unique and that's like an unfortunate thing, but it's also like uh you it all the LMs because they're not like you know the same thing every time, right? Like uh that's a positive for that, right?

SPEAKER_02 28:49

Yeah.

SPEAKER_00 28:51

Huge hole I see missing is kind of like that learnability aspect of things, right? Like something that you could just attach on to your experience that just like is watching you and your behavior with it, right? Like, and it can inject like you know, that behavior into the context as you're using it. Yeah, and like that's hard to do, right?

SPEAKER_01 29:12

Like, yeah, it's hard to even watch it, let alone persistent, right? Right.

SPEAKER_00 29:17

So that's uh and so I've been messing around with like some sub layer like actions, right? Because they they can watch files and and I feel like this is actually something I've built so many times is just like a log, right? Like what has been done, just like just keeping track of the log. I haven't actually used it, right? Like I don't know what to do with it now that I have it, but like, you know, I I keep building them and like eventually be like, you know, if I could just like find how to like introduce this into however I'm using it, like there's value there, but like I don't know if that's the right way to do it, right? Like, but like I feel like there's something there where it's like, okay, like your use of it is gonna like train it somehow at some point.

SPEAKER_02 29:58

Yeah.

SPEAKER_00 29:58

Do you think I'm off there?

SPEAKER_01 30:00

No, I don't I'm actually really interested to see this now because I think that there's I do think there's something there. And I think you know, we've had this. Yeah, I mean, I guess I haven't heard it this year, but uh you know, last year people were always talking about training LLMs, and uh yeah, to a large extent that was confusing.

SPEAKER_00 30:19

Maybe maybe we're just using like too dangerous of a word training, right? Like that's very specific to this world, right? Like it's almost like not like we're training necessarily, but like providing some you know backdrop to experience, right? Like AIs are becoming like we've made these chatbots, and it's like now, okay, well, they're all the same chatbots, and now people are trying to like augment the context to like change the experience, but really we're like at that experience, right? Like, and like how okay, how do we rework it to like change the experience to a more personal level? And like, yeah, I don't know. There's like that's it's really uh really interesting to me.

SPEAKER_01 31:01

Yeah, it is, and it's also you know what we've found is you know when we try to you you try to bolt things on to the LLM, right? Either additional context or additional instructions. And the thing is that these LLMs are changing really fast because that's where most of the money is going. And uh and you really we found that you really have to keep a lot of loose coupling, like you really have to structure your code in such a way that the things that you're building around, like especially the like the blast radius of your LLM, that stuff has to be able to just be you know deprecated completely or changed out really fast. We were on our second agentic agent orchestration library, and I'm pretty sure my team's gonna tell me. And it's just you know, these things outlive their usefulness really quickly, and uh so you gotta be able to move and change really quickly, too.

SPEAKER_00 31:47

Yeah, I feel like that's not unlike all software, right? Like, yeah, you know, a lot a lot of people even hit those limits with Rails, right? Like Rails is a great like kickoff utility like framework. Like you have something on the web that you want to accomplish, and like Rails is awesome for that, right? Like, and you can build an incredible business out of that, and like you know, at some point you may see yourself being like, well, you know, Rails has out, you know, our our like business processes have like outgrown what Rails had offered us originally, right? And so like people do like find something else, even within the Ruby space, too, right? Like just building something themselves that fits the business better. You know, Stripe is a famous like example of this, right? Where like they are probably one of the biggest Ruby shops that's not Rails, right?

SPEAKER_01 32:38

Like, yeah, right.

SPEAKER_00 32:40

And like I feel like that's a similar thing with all these like AI frameworks or you know, things that happen is like people are now like outgrowing them to the to the custom point, right? Where like we now have custom AI like software that we're like focused on how do we build that best, and like that's always gonna change, right? In the same way that like software, you know, frameworks and usage changes too. Curious if like that normalizes in the same way though, right? Like, we're at such like a a hockey stick, like in the graph of advancement, right? Like, seeing all these like coding agents and stuff like that that are like coming out, like it's difficult. How do you gauge like alright, this makes sense to pursue, and this doesn't, because we know like it, you know, it's probably not gonna be worth it, right? Like, it's gonna be gone like in a month, right? Right.

SPEAKER_01 33:36

I spent a little while on the sideline and my personal opinion is dug in start building start you know to sort of take advantage of the new tools as they come out. You don't know how you know the next big thing is, unless you already have worked out everything before. So that's been my take is to get our get our team just deepen the weeds of of AI, specifically how AI automation. I think like everything else you know, so it's repeatedly test kind of the use cases that are important to do with different all times, different prompt, whatever. Um, and you know, and be ready to shift and shift quickly. But I think one thing is that um you know you really have to build in a way that can take advantage of what's there without sort of obfuscating what's there. And that's I think another big advantage for the huge AI generation tools that are out there right now, because it's just a thin wrapper. So you kind of don't need to, and some companies are doing some really cool innovation, but you kind of can let the LLM lead the way. That's that's we're still kind of keeping that as a core concept because again, it's where billions of dollars is going. So it would be foolish to just say, well, we're gonna build separately from that. You know, we want to build with it, but make sure that we're always able to take advantage of those innovations.

SPEAKER_00 35:02

Yeah, I agree with you there. It's uh, you know, you definitely have to chase it, which is kind of exciting to be in the industry, right? Like uh you gotta see what works for you and what doesn't because it yeah, you never know what could stick for you. And and so I'm curious, like I'd like to transition us into like you know, the Ruby space specifically for like all this tooling, right? Like because there's a lot of talk, like you know, Langchain has like their whole stack that just like is kind of like the Rails for AI. Has been for a while, like where you oh just go and like build stuff and it locks you into that Langchain experience and like people outgrow in the same way Rails do.

SPEAKER_01 35:38

Yeah, yeah.

SPEAKER_00 35:39

What is what are like some of the Rails equivalents that you see uh that are kind of like setting in for some of this, like hey, if you want to build with AI stuff, like this is a great starting point. Oh, that's a good question.

SPEAKER_01 35:51

Well, the easiest way to be wrong on a tech podcast is to tell you what the next Rails is gonna be.

SPEAKER_00 35:57

But uh There are no right answers on this show, are you kidding?

SPEAKER_01 36:02

Okay, yeah, yeah. I think I'd say that um, you know, I actually I I like Scott's Sub Layer a lot. I would love it if that was the next library that we moved to. I know it's a bigger shift because right now we've used Crew AI for about three or four months now, which is open source. I think they have um I think they have like a premium tier that is more full service, but we haven't used it. But you know, you start to see some of the cracks in these tools, and it's just because um I think, I mean, I I guess I'd like to think that most people aren't using them as an extensive use case as we have. Now, there are some and some that are far more complex, but I'd say that we're probably on the 80 to 90 percent, you know, more complex use cases when it came to uh Langchain. We still use Langchain to evaluate and like find out how much we're spending uh every day, which is alarming. But um but we don't use the orchestration anymore because we think it's um well, it's just it became very difficult to set up. It be it wasn't really fitting, uh again, like you said, it wasn't really fitting our use case. I would like to move move ourselves to Sublayer. Uh Scott said he would uh come and port it over for free, so when he does that, we'll move for sure. And uh I guess I'm not sure beyond that, like what the next big thing will be. What do you think?

SPEAKER_00 37:21

Yeah, I've definitely used Sublayer uh obviously heavily. I don't know if that's just because that's what first came out or not, but it does solve most of the use cases I've seen for building with AI in a very rubis way. Maybe if I was new to Ruby, there would be other things that might be better, uh, because I don't have to know as much Ruby. I don't know. Hmm. Yeah. I I don't know if that's the same use case for uh Langchain though, right? I feel like m many people maybe don't know Langchain either or Python.

SPEAKER_02 37:47

Yeah, right.

SPEAKER_00 37:48

You know, going into it. So um Yeah, it's hard to say. I I feel like, you know, Shopify just announced their roast framework, which looks pretty promising as like a similar command line workflow orchestration system. I feel like there's still like a lot of like people missing the boat on like benchmarking and evaluation of like what it's doing. And yeah. For that reason, I'm very hesitant with any of them.

SPEAKER_01 38:20

I see. You mean the tools themselves are not benchmarked?

SPEAKER_00 38:23

Right. The tools themselves, like I feel like if you want to build like a quality AI system, like evaluating what it's doing is like one of the highest priorities, right? Like if you're not testing it in a uh like an automated fashion, at least some way, and like scoring and judging it uh and having that feedback loop, it's very difficult to like get any like long-standing quality use out of it from a business.

SPEAKER_01 38:49

What's the score that you're looking for?

SPEAKER_00 38:51

I that's up to you to make, right? Like, but like having tooling in place to like make those up uh are kind of like really important. So like you know, there's lots of vendors that offer this service as like a sideways product, and then it always creates this issue of well, then you have all these tools that then have to connect to it, uh, right, and so like you're then uh dumping all of your logs at right these services, and then having to like massage that workflow, right? And like, so how do you then get the feedback mechanisms right from your app into those? And then it becomes this whole like massive orchestration where you're trying to like just work with vendors all the time uh to solve something that should be built into the tools, to be honest. And I feel like it's often like missed as like a framework, it should almost be like the first choice, right? Like, here is the uh evaluation system or or metrics builder for a an AI framework, and then starting from there, then all of the features start getting built into that. And I feel like Sublayer provides all the tooling to make that happen, it's just not like automatically there, right? Like you kind of have to like make your own system out of it, which isn't bad. Uh it's just like more modular and more like you know, you have to do it.

SPEAKER_01 40:07

You have to hand it more of an investment, yeah.

SPEAKER_00 40:09

Right, which isn't, you know, uh for for Rubyus is actually nice because we like to build things ourselves and of our own, right? Like, so like it's I'm split, right? Like, because I would like to have these things just automatically work for me.

SPEAKER_02 40:22

Yeah.

SPEAKER_00 40:23

And like and defined, to be honest. Like Rails is very like opinionated. I would like an opinionated, like, this is how you do it. Yeah, I can see. Yeah, I can see. But having the flexibility is also important in an AI framework, right? So the active agents library by Justin Bone is like super promising too, right? Like it's built into the Rails ecosystem, so like you get all of the familiarity, and it's like plug and play, and like that the aspects feel like testability is something that you could just use Rails for, right? And be like, okay, I expected to do this thing. I feel like though, with a lot of these like agentic stuff that's coming out, it's like, well, how do you like then do the integration test, right? Like, there's so many variables in that workflow, like it's not like you're just like input to output, it's like right. Yeah, just I'm thinking about VCR trying to Yeah, right, like input and then side effects, and then right, like, well, how are you measuring those side effects, right? Like, and like that's very challenging. And I feel like even some of these uh valuation vendors like have a hard time capturing that, right? Like, how do you measure the side effects within your business processes, especially as you start to hook everything up to it?

SPEAKER_01 41:38

Yeah, I know. I think you know the active agents one is something that's exciting to me too, and we'll have to have them on the show to uh active agent folks to I would like to try it. It's um like the the Rails part of our application is still a little bit distant from the agents, but it's getting kind of closer and closer every day. So it could be an exciting way to you know kick the tires.

SPEAKER_00 42:01

Yeah, I would say next to those, like, you know, Obi Fernandez, he's killing it like the whole Ray framework is awesome. I've used that like just I really like the descriptive, it has its own DSL, so like I really like the descriptive nature of like the objects that you build with it as like a okay, this thing has a bunch of steps in it, even that you can hook up to each other, and like you declare the prompts very specifically within the class structure. Yeah, there's a lot of parts of it that I really that like really snap in play too.

SPEAKER_01 42:32

Well, so tell me a little bit a little bit about it, because I don't know the Ray.

SPEAKER_00 42:36

Yeah, so the the Ray Library R A I X, it's very declarative in like okay, you're gonna have, you know, let's say you're generating like an you're parsing an invoice, right? And you want to do something with that invoice, like you make like some kind of like you know, invoice, you know, prompt and declare it as like a class, and it's like actionable, and you can say, okay, like it's gonna do this prompt, and then it's gonna use the output from that to do this one, and so you can chain a bunch of stuff together, which is nice. Uh, and it has like a lot of the the niceties where you can like attach arguments to the prompt and declare that in and like declare your tools that it can use, and so you can abstract kind of like an agent aspect. I don't know, I don't think it's actually called an agent, but like Yeah. You can you can create these objects kind of like just regular Ruby objects that have these attributes that can be reactive based on what the input is.

SPEAKER_02 43:32

Okay.

SPEAKER_00 43:32

And it it's a really nice like utility to to work through, and I like the er the ergonomics of like it almost has these like then steps. So like after it has a response, then you can do something like that. Uh yeah, and the chain ability aspect, yeah. I really like it's like active record style, like where you where you can chain a bunch of methods off of the same thing and use them together. That's cool. I'm gonna check that out. Yeah, but again, like I'm just experimenting, like I haven't built a product on it, so like I don't know. You know, a lot. term how that pans out, but you know he obi's building quite a lot. So like yeah I trust the framework survives, especially as Shopify starts to embrace it, uh which they have. So that's that's they have yeah.

SPEAKER_01 44:12

Yeah I had a uh had a talk with Obi and a couple of the other guys at Shopify not too long ago. We were just kind of comparing notes on you know how you know structuring our programs and structuring the applications and they just it's so funny because they're just like you know we've said well we have had different experiences with Claude versus at that time I think probably ChatGPT 4.0 and you know one of them at Shopify was like yeah we've got you know we're talking to Claude and and ChatGPT about like what they're doing for Ruby. I'm like that sounds nice when they're not taking my phone calls. Yeah but you know if you're the biggest Rails application in um in the world you know you can get them on the phone and uh maybe so I'm like yeah you know they're gonna advocate for us please that's awesome.

SPEAKER_00 44:54

Yeah I uh I'm excited to see what comes out of Andrew Kane this year. Yeah he he's just like keeps pumping stuff out for the Ruby world specific specifically the stuff that I follow for fine-tuning I keep waiting for the day where I I could just like uh switch over a bunch of stuff I'm doing. Yeah it's a it's I know just use Ruby uh right now I have these stupid like you know python scripts that I'm calling shelling out to basically which is not fun but you know work with what you got yeah exactly I'm curious like uh because you mentioned you know you were using Python for some things like is that still the case like how does that how is that transition working or integrating with with Ruby like what are the challenges you have left there?

SPEAKER_01 45:40

Well so basically this was a this was a Python app for about the first three months uh until I was kind of like they were deep in the weeds you know they're working on giving proper context on you know on getting the the proper automations in place um just just for the agent and right at that point it was just a CLI and and I think at some point yeah we realized hey you know we're gonna need a we're gonna need an app web app that shows what we're doing and so like over a weekend I was like alright we're you know Rails eight like let's go and you know it spun something up and the whole point was like hey this is just a show this just shows what the LLM does and so I it was Ruby but it was you know it was a Ruby application but really it was just Ruby it was just a showcase point. And then it started to become like oh actually yeah we're moving away from anything like any kind of interaction on the command line like we want to we want to have this thing working end to end and continuously working. And so then that continuous part became well it's Rails to GitHub and pro possibly GitHub actions and there's some Docker in there and then you know there's the uh the CLI at the other end. Yeah. And this just and like when we saw that and uh we can get into this on the next episode like we did a lot of like we did a lot of things that probably a few years ago would have been considered big upfront design and today because AI just like spits out like ERDs and you know and like mermaid charts in like two seconds it's like now like oh we've got like really established stuff and it doesn't matter if it changes.

SPEAKER_00 47:18

So then like when we started we looked at this thing is it yeah mermaid's good oh yeah I mean because even making changes you just be like oh move this node over here like and it just does it like perfect. It's fascinating.

SPEAKER_01 47:29

I like the first time I saw it I was like what did you do? You spent like a week doing this and then you know it was like no it took like five minutes. I'm like all right awesome let's keep doing it then but yeah so like when we realized that it was like oh actually there's like all of these moving pieces and we didn't want to write a bunch of Python and manage that right so it became then all of a sudden that little you know UI application became like a focal point for the I'm trying to use a word other than orchestration the like the managing the integration points across everything. And uh and so now the CLI is still really important and that's written in Python but it's becoming a smaller and smaller part of the application not because we're making it smaller but because the rest of the application is growing.

SPEAKER_00 48:10

Yeah that's exciting to hear it makes me wonder because like crew AI has a heavy Rails for something. I will have to have them on to find out what that is. Yeah I'm I'm curious like how the the lines start to blend right because uh I feel like every company is like especially that got in early python had to be at the forefront right yeah yeah a lot of progress but now yeah now that it's stabilizing like Python's not great for web yeah I know it's not a lot of missing pieces you know even the Django like it's funny I'm friends with a lot of Django folks and they're like you know they try and look to Rails like oh how can we like integrate that it's like every day like how you know how do you do this in Rails like in Python but from Rails.

SPEAKER_01 48:52

Yeah I know I've I've experienced that too and um it do it does get really complicated. Yeah I would never start uh a web application like I've built web applications in yeah C sharp it's it's kind of annoying I've done it in Java it's it's fine. But yeah Ruby still feels like the best it's it's hands down and the web's not going anywhere I don't think anytime soon. So we're gonna keep building for the web.

SPEAKER_00 49:19

Yeah totally well I think we're we're hitting our limit here on time. Otherwise we're gonna keep talking all day long.

SPEAKER_01 49:27

I know to be honest this is good though.

SPEAKER_00 49:29

Yeah I mean uh if you're out there listening you know we have a lot lined up here uh we're gonna start having on guests uh every week if we can uh you know we're gonna explore the depths of everything that's happening in the AI community uh and you know try and focus on Ruby as much as we can yeah uh you know as things progress. A lot happening and you know it should be exposed and find out more about you know how people are using it because like everybody's using it a little bit different and I'm always you know kind of impressed anytime I I see somebody talking about what they're working on like whoa like yeah you can do that you know and like uh we gotta get more of that you know we gotta get more of that and we have to make it less shocking when we hear about a new cool thing right like you know you come here and you're on the ground floor and then you know and then everybody knows.

SPEAKER_01 50:18

Right. That we can all be on the same page.

SPEAKER_00 50:20

Yeah totally all right Joe was it was great talking to you today. Uh yeah you too I feel like maybe we should close out uh you know just like sharing one thing that we found interesting recently uh you know in the AI space uh that you're pretty excited about uh um all right you sit you first I I can go first I've been messing around with uh code agents specifically Hugging Face has this thing called small agents and there is a a deep learning course on building these code agents with their hugging face small agents super cool it is in python but it like really eye-opening kind of what you can do when you empower an LLM with uh its own like ability to code and make use of coding in general so I highly recommend checking this out it's uh it's on deep learning one of Andrew Ng's courses building with code agents with hugging face small agents very cool I actually um I wanted to get that uh I wanted to highlight again that that Ruby OpenAI official project that I'm having trouble finding because I loved the woman who presented was like a no nonsense Rubyist and now I'm trying to find what it is.

SPEAKER_01 51:35

Give me a moment here. There we are yeah so I just I liked the the openai ruby it's openai slash openai dash Ruby it's good software it makes a lot of uh it makes a lot of use of sorbet which is a little contentious around deaf methods but you know in a real way um it tries to to give you some static typing or some benefits of static typing it's uh it's insanely fast it's it's a really good place to start if you're trying like most people you're just starting out you're trying uh you're trying to integrate with uh you know ai in an interesting way that's not through just a a chat this is a good a great place to start yeah that's awesome I'm I'm happy to see that come up too yeah well this has been really great Joe we gotta do more and more of this yeah we're branding ourselves the Ruby AI podcast we'll see how that sticks I feel like we're in for a lot of fun times ahead of us I know you'll be at RailsConf coming up here I will be at Railsconf I'll I will also be there awesome so if you're out there listening come and see us we'd love to talk about AI and and Ruby and uh who knows maybe we'll we'll have a show uh I don't wanna you're speaking so I I don't want to like put any pressure on you also do a show yeah I think it'll be fun we could we could right uh I feel like that could be a lot of fun yeah but yeah stay tuned and we'll be releasing more and more yeah stay tuned all right thanks everyone we'll talk to you soon