Valentino Stoll (00:01.742)
And we are live. Hey, everybody. Welcome back to another episode of the Ruby. Sorry. Yeah. The Ruby AI podcast. Still getting used to it. Today, we are joined by Joe Leo here, my co-host. And we have a very special guest today, Justin Bowen. Justin, you want to tell everybody why you're famous here and why we've had you on?

Joe Leo (00:11.423)
Yeah, you gotta ride.

Joe Leo (00:19.42)
Hey everybody.

Justin Bowen (00:24.216)
Hey, thanks for having me.

Justin Bowen (00:30.372)
yeah, I don't know if I'm famous yet, but looking forward to it, I suppose. that's good. Yeah. You know, I've been a Ruby developer for 17 years now, and I only recently started going to conferences during the pandemic. won a ticket to RubyConf, from Mike Delasio. and then like just on Twitter, some thing that I entered did the virtual thing.

Joe Leo (00:33.33)
You are Ruby famous. It's a special kind of fame.

Justin Bowen (00:59.82)
And then I did a talk 2021. That was my first talk virtually on Rails and computer vision, which is something I've been doing for about 10 years now. So yeah, excited to talk to you guys today about Active Agent and whatever else we come across.

Valentino Stoll (01:19.1)
You know, that was a great talk in 2021. That was actually where I first saw you personally in the community. And I just loved that talk where you had like some agricultural imagery where you were super composing the images together to find anomaly detection for plant health and stuff. Yeah, super cool. I didn't even know machine learning was in the Ruby space yet at that point.

Justin Bowen (01:25.218)
Yeah, that's right.

Justin Bowen (01:38.381)
Yeah.

Joe Leo (01:45.939)
you

Justin Bowen (01:47.215)
I mean, it kind of wasn't. And that's one of the questions I had in the Q &A is like, why didn't you do this in Ruby? I listened to your guys' first episode and I think it's more similar to what Joe has been doing using crew AI as his like Python AI runtime. I mean, we're here. But yeah, like I was doing Python and OpenCV at the time, but Rails has always been my like...

Valentino Stoll (01:49.734)
Right.

Joe Leo (02:04.359)
Man, I didn't know we were gonna get into that right away in the first two minutes. Hey, so I actually had to look this up.

Valentino Stoll (02:06.47)
Ha!

Justin Bowen (02:16.022)
Ruby is my love language, I like to say. I have a Ruby in my wedding ring. But yeah, I've always had Rails backends for all of these computer vision applications and AI projects that I've done. So yeah.

Valentino Stoll (02:30.322)
Yeah, that's awesome. Yeah, so I mean, that really set the stage for me personally, following your content, because yeah, your talks since then have just been like, spot on to a lot of things I'm interested in as well. And Active Agent, awesome framework. I think coupling AI to the Rails framework is a really smart choice and something I think the community will see how powerful it is and valuable soon enough.

I would love to dig in here and kind of like get your high level, like what even got you on this path to begin with.

Justin Bowen (03:01.646)
Yeah.

Justin Bowen (03:07.822)
No, that's how far do we want to go back? I mean, like, I honestly, it started with video games and playing video games with my dad and getting into programming and rails is something I found, uh, in 2007, actually, um, OB Fernandez is, uh, the rails way was my first purchase on Amazon. found out recently in like an icebreaker. Uh,

Valentino Stoll (03:11.334)
You

Justin Bowen (03:37.027)
An icebreaker for a company I used to work for, I guess about a year ago, almost. And, I joined the company as a staff AI engineer. And I thought I would be doing more Rails and Python because that's, they were Rails monolith and that's what I had been doing in the past for AI. And I ended up doing a lot of Next.js there. so like little Next.js apps built around the Rails monolith as a service.

and using the Versailles SDK. And, you know, I, it wasn't what I thought I would be doing. And I was frustrated we didn't have that kind of tooling in Braille's ecosystem. You know, there are interesting frameworks in Python. Like, I'm sure everyone's familiar with the name Lang chain and what they're doing and crew AI Joe, I you're using even those like

Joe Leo (04:12.592)
Yeah.

Valentino Stoll (04:12.815)
you

Justin Bowen (04:33.61)
In my opinion, pale in comparison to the full stack integration that you get with first cells, AI SDK. And that's kind of what I'm trying to do with active agent. So active agent is the framework and the gem and an active agent class is really just an abstract controller. And so the concept of agent oriented programming is something that I am trying to dig up.

Apparently it's like a 30 year old term, but the way I think about it, yeah. The way I think about it is like object oriented programming, except instead of our classes representing lifeless objects with properties and functions or attributes and methods, if you will, they can actually have more of a persona and perform these actions. And, know, I

Joe Leo (05:04.699)
Really?

Joe Leo (05:25.809)
you

Justin Bowen (05:29.164)
Honestly started this as a joke last year at Sin City Ruby. just joked about the name, after Obi's talk at Sin City Ruby. was his first talk in a while, talking about the content for his new book that's out now. And Landon Gray was there, Irina Nazarova and Marco Roth. And so those three are definitely big influences on this existing.

Joe Leo (05:57.093)
Hmm.

Justin Bowen (05:58.447)
And after teasing that, I didn't really think about it again. I went to Ruby Kaigi and while I was in Japan, this is my first Ruby Kaigi. Um, and while I was in Japan, rails, world tickets sold out instantly. uh, I got a message, uh, from Rena and she's like, Hey, did you get a rails world ticket? And I was like, no, it was like 4 a.m. when they went on sale and they sold out like immediately, just like they did this year.

Valentino Stoll (06:13.266)
That's crazy.

Joe Leo (06:13.945)
Yeah.

Justin Bowen (06:28.46)
I was also in Japan. did score a corporate ticket this year that I paid for, but arena, think, I'm not sure. think she had extra ones from CFP submissions or she purchased an extra one and she kind of used that as a leverage. she, she said that, I can have the ticket under one contingency that I actually build this. Cause I had talked to her in the past.

Joe Leo (06:34.362)
Yeah

Joe Leo (06:47.578)
That's great.

Justin Bowen (06:54.882)
Since she's moved to, guess like since before she moved to San Francisco, we've become good friends and spend a lot of time, chatting about open source and those ideas that I've mostly had in my head. And some of them I've built out like the computer vision for watching plants grow. did make a couple of Python packages out of that. And, I know at least one person used it. but I was teasing about making a.

Joe Leo (07:21.328)
Excellent.

Valentino Stoll (07:21.723)
You

Justin Bowen (07:25.006)
a computer vision application framework about two years ago. And I never, I've talked to her about it a bunch and never went through with it. So here we are today and the gem is published. There are better docs coming this week actually. And I'm doing a live coding session at SF Ruby tonight. yeah, long winded answer to a simple question.

Valentino Stoll (07:35.762)
You

Valentino Stoll (07:46.278)
That's awesome. Yeah, you know what I made? I did make a little app for us to experiment with active agents to kind of just intake our episode. I'll message you later about it. But yeah, really, really exciting. Like I just had it like, you know, we get email intakes when guests, you know, accept our offers to come on or beg when we plead people to come on. Yeah, I just dumped the email content in and extracts like the date and

Joe Leo (07:47.066)
grade.

Justin Bowen (07:55.884)
On that go.

Joe Leo (07:57.488)
Mm-hmm.

Justin Bowen (07:59.63)
Yeah, yeah,

Joe Leo (08:08.624)
Mm-hmm.

Valentino Stoll (08:15.858)
questions, topics, it does a whole bunch of research for us, which is nice. Yeah, it's super straightforward to use the Active Agent for that. really, yeah, really excited. I mean, I'd love to take a step back again, like, just like, because this is the year of agents, supposedly, right? Like in the AI world, right? So they say, so like, and you mentioned, you know, agent-oriented programming, like there's a bunch of like,

Justin Bowen (08:21.496)
Awesome.

Justin Bowen (08:25.614)
It's great to hear.

Justin Bowen (08:36.558)
Yeah.

Valentino Stoll (08:45.71)
agentic stuff coming out like, you know, hugging face has small agents for coding agents and, you know, there's the agent to agent protocols that Google announced, right? So I mean, like, what what is an agent to you? What are you building and driving? What's your driving force for like, abstraction around this like concept of an agent?

Justin Bowen (09:09.282)
Yeah, I think it just goes back to like MVC. Like that's something that really drove web development and figuring out what, again, like thinking about a computer vision application framework, like what is the controller in this situation? Like models and overloaded term now, but like, you know, what are the data models that we need to interact with? And then like a view layer is something that I feel like a lot of people have just been.

Overlooking except for Vercel. now Maestro built on top of Vercel is expanding on that idea of agents that can interact with users with generative UI rather than just like everything being a chat. And so, yeah, I think for me, an agent is a controller and the context is that model representation of whatever the agent's interacting with and

It has access to ActiveRecord as well, since it's just a controller, like ActionMailer is just a controller. And it also has access to ActionView. So you can have templated prompts and templated views that can be rendered to the agent or rendered to the user. And yeah, it's the C in the MVC.

Justin Bowen (10:34.946)
But yeah, I think agent is also an overloaded term, you know.

Joe Leo (10:40.15)
Yeah, it absolutely is. It's already annoyed me.

Justin Bowen (10:44.236)
Everyone's excited about coding agents. I think it's interesting. think a lot of it's hype. This, this AI hype cycle has actually reminded me a lot of crypto. I've been doing AI development for a decade now and, yeah, I did a little bit of work for web three clients in like 2017 to 2019 and worst clients. Thank you. But, but so like, you know, there's, think.

Joe Leo (11:07.564)
Yeah, didn't we? We don't have talk about that. We all do.

Valentino Stoll (11:10.672)
You

Justin Bowen (11:13.932)
A lot of people are still trying to figure out what the value is here. There's definitely more value in this than crypto. I don't know if that's replacing developers. I don't think that's like where the value is. Code generation is interesting. I think there is value in that, but it's not to replace us or junior developers. I think that's really short-sighted and more of a mask for mass layoffs than anything to pad Microsoft stock and others that have.

Valentino Stoll (11:41.106)
You

Justin Bowen (11:42.688)
made layoffs and claims of like meta saying 30 % of their code is AI generated. Like, good luck.

Joe Leo (11:48.096)
Yeah, I totally agree with you. You know, when I asked this question to somebody I was talking about this very topic with yesterday, he's like, well, you know, I'm not so nervous for me, but I'm nervous for the junior engineers because they wouldn't have jobs. And I'm like, okay, so do Ruby development just stop when we retire or die like when the old guys are done? It just doesn't make any sense to me. That's not really how the industry works. Sometimes industries go away, but it's not like that.

Justin Bowen (12:12.846)
Yeah. And I mean, even with computer vision models, like I started with YOLO V3 back in like 2015 and the last production application I was working on YOLO V8 in 2023. So like right at the cusp of like GPT-3 coming and like becoming mainstream. But it's, it's like the Ron Burgundy quote, like 60 % of the time it works 100 % of the time.

Valentino Stoll (12:36.889)
You

Joe Leo (12:38.317)
Yeah, it sure does.

Justin Bowen (12:40.302)
It's like when we were detecting cow faces, you know, we could detect maybe eight out of 10 cow faces. When we were identifying cows, was like maybe 60 % of those 80 % of the cow faces that we were able to identify. Watching the cows, it was actually pretty easy because it was just like optical flow. Like once we had a cow face, we could see their mouth moving and where they were. But, uh, yeah, I think.

Joe Leo (12:51.053)
Hmm

Justin Bowen (13:06.668)
There's still, I think one of the important topics you guys touched on last, or like your first episode was evaluation and like, how do we make sure these agents are doing what they're supposed to in production? So.

Joe Leo (13:19.511)
Well, that's an interesting topic too. I'm curious about that. Obviously I'm thinking about things from a testing perspective. So it's tech week in New York right now, is fine. If you're listening and you run tech week, we love it. It's a lot. mean, if talk about hype cycle though, it is just people talking about their AI startup and why they need millions of dollars. And I was in a room with a...

Valentino Stoll (13:20.699)
Yeah.

Justin Bowen (13:46.979)
Same.

Joe Leo (13:50.957)
They were interviewing one of the chief scientists at Databricks and it was a really interesting conversation, but he asked the crowd, it about 100 people. He said, okay, how many of you are building with AI? And every hand goes up. I was like, okay, how many of you have a testing process in place to assert or refute how well your AI is working in production? And three hands went up. Mine was one of them. So there were only two other people in the room that even thought about that.

enough to give it some credence. So I'm curious to know how might we, I mean, two things. Does Active Agent make this any easier? Right? The idea of being able to test something non-deterministic in a production setting.

Justin Bowen (14:36.13)
think it can make it easier. think monitoring for evaluation will be easier because it has like similar instrumentation to what we're used to with an action controller or other experiences in Rails. But I do think that it's something I've actually thought a lot about in the last decade is like, how do I test these things in production? Because when we're...

doing transfer learning on a vision model and the vision model is supposed to detect surgical objects in an operating room. The original model that we're doing transfer learning on is trained on the Coco dataset, common objects and context. Another word that's very overloaded is context. and like these models perform well, in a specific context, but you change the operating room. Operating rooms are like chaotic. Dairy barns are very like.

simple, dirty, but simple. And the context matters and it took time to evaluate and annotate data in each of the operating rooms in order to get it to work across operating rooms and across facilities with different styles of operating rooms. And it's important to have a ground truth for whatever you're doing, whatever the context is.

and be able to write tests for that because you want to know like how well does this perform right now, whether it's like this prompt with this agent that has these instructions and this context. And then what about my next iteration of that? So I think right now that's less what I'm worried about. That is though longer term what I'm planning to build out as part of.

the business model for Activision. And I know, Valentino, you were kind of on the fence about that idea. That's how Langchain makes money through LangSmith is their prompt evaluation and editing platform. I don't know, Crew AI's monetization is probably more like enterprise package and services.

Joe Leo (16:45.301)
you

Joe Leo (16:50.942)
It is enterprise. don't pay any money to Crew AI. And I feel like I'm being, I feel like I'm being, I feel a little bit like I'm being nailed to the cross over Crew AI. Look, I just want to state for the record that, I don't want to state that, no, it's pretty good. We actually started with Lang chain, quickly moved on. didn't have, now we use Lang chain exactly what you just mentioned. We use the evaluation system because we want to know how much these things cost.

Justin Bowen (16:55.404)
Yeah, that's a big issue.

Justin Bowen (17:01.756)
I'm sorry. No, hey, it's fine.

Joe Leo (17:18.124)
We moved on really quickly from the agent orchestration and moved into Crew AI. At that time though, it was a CLI, Phoenix was a CLI and we quickly moved on from that. And now it's more and more Rails and less and less Python, but there's still that Crew AI in there. But this is interesting. When I think about...

People that are, now people are already using this thing. There's hundreds of stars and there's forks and there's activity. But I think about what people are listening to the show and maybe like, okay, I want to start, but I have no idea, like how am going to put something into a spec or into a unit test and assert that what is happening in this active agent is actually working. I guess it's an integration test and not a unit spec where you can kind of mock everything out.

And I'm wondering if you've given thought to that or if in your professional background kind of implementing it if you're helping teams do that.

Justin Bowen (18:25.112)
Yeah, I think again, it's about knowing what this like ground truth context and scenario are, like what the objective of the agent is. And then you do have to run it against a model. And if that model changes, you need to run it again. So if you're using VCR, I heard you guys mentioned, you're going to have to record a new cassette because you want to know like, and that's something I would recommend like as part of the CI CD process, like.

Joe Leo (18:31.455)
Yeah.

Joe Leo (18:43.292)
Yeah. Yeah.

Justin Bowen (18:52.258)
deploying it into staging, maybe actually run it against a model, because it would be worth the expense to do it. So that you can validate like, is it actually performing as we expect or better? And that's something that's challenging depending on the context. Like where I'm talking about counting objects or counting cows, it's pretty obvious. Like the ground truth is there's eight cows in this picture. We should be able to detect eight cow faces. If we can detect seven right now.

Joe Leo (18:59.082)
Absolutely.

Joe Leo (19:07.977)
Mm-hmm.

Joe Leo (19:15.903)
you

Justin Bowen (19:20.204)
we should be able to detect seven or more on future deployments. And so like, that's the scenario that you're testing against in these integration tests.

Valentino Stoll (19:22.812)
Yeah.

Valentino Stoll (19:31.132)
Yeah, you're so right on there's so many different aspects of like of your agent usage, right? Like that can change the end result, right? We talk about determinism and non determinism and like so much of how you build the thing, right? Like can make it less deterministic, right? Like the tools that you use, even the number of them, like a small word that you've changed, you would have moved a period in a different position from the prompt, right? Like

If you provided the same content but in a different format, there's all these variations that can affect it. And the ground truth is your end result that you expect the behavior. And as agents start to move, they perform actions. So how do you monitor those actions is another aspect. it does make me think, so I watched Matz's keynote from the recent Kayagi.

Joe Leo (20:09.279)
Yeah.

Justin Bowen (20:28.302)
Mm-hmm.

Valentino Stoll (20:28.794)
And he was like all AI, right? And like his underlying concept is, yeah, I was very surprised myself. I loved it. But one of my key takeaways that I think I'm a heavy alignment on is like the concept of like, you know, we're at this critical moment of like, either we're gonna be building things for the AI or we're gonna be like, you know, building things.

Justin Bowen (20:32.318)
surprised about that.

Valentino Stoll (20:57.898)
Like who's the master, right? Like the AI or us, right? And like, we're kind of at this critical point of like, are we just building things because like it's what the AI demands? Are we building things so that we can demand things of the AI? And like the commonality of all of these things is like the human interface element, right? And like the basically the natural language is like, is that like moment of instruction. but also like

you know, to your point of like evaluating, gathering those ground truths and annotating and like, you know, finding all of the truths of your data that you expect of the context, right? So like if you're, if you're in a stall with cows and like you want to know how many cows are in there, like, well, okay, well, what kinds of cows, like there may be other important details that like, you can't just, the AI won't be able to gather from context of an image, right?

but like you being more descriptive may help it infer the generation and result better. And so like, I'm curious, like, where do you see the abstractions heading for like pulling out and drawing out those ground truths during the process? And like, what point in the process do you start like thinking about those?

Justin Bowen (22:17.484)
Yeah, I think it's a challenging topic because it goes back to people are still trying to figure out what to do with these things. think once you know what your use case is, like, and there are a few, good examples that I know, people are posting about publicly, like, are you guys familiar with Jordan Trevino? he runs an agency, Telos Labs, and they're working on a

Joe Leo (22:46.744)
I do know teller slabs, okay.

Justin Bowen (22:48.846)
a product called SimpleDocs and it's like legal document review redlining. I also know Kelly Sutton is doing something in like for, I think it's called Scholarly and it's for like resume reviews for academic candidates for roles at universities. And so like these examples, like it is a little more straightforward. Like you have

whatever legal document and then whatever expectation of what these red lines should be or whatever expectation of being able to parse out this unstructured resume data into some structured data set. So having that, it's still like you need to have a collection of these ground truths in order for it to be, you know, useful and valid because

Again, it's all about the context. If it's a different type of resume, it might be parsed differently, different type of legal document might not see certain things that it's supposed to see. And then that gets into like, well, how do you make it see those things? You have to change the prompt probably. And then once you change the prompt, does it still do what it was supposed to do in the beginning for your other ground truth data? And so that's where I think, again, it's a complex problem and

Valentino Stoll (24:08.242)
you

Justin Bowen (24:17.048)
I think that's part of why a lot of these frameworks haven't spent as much time on those problems. Like Vercelles AISDK, for example, they show you how to tie in with like open telemetry and open LLM tree projects. Like there are services that are specifically about evaluation tooling, like New Relic, but for agents and prompts. And I think that's kind of the space that I'm interested in for.

the future of Active Agents as a business. Right now, Active Agents, I'm just looking for consulting work, doing contract development to help build out the framework and build it into people's products. yeah.

Joe Leo (24:53.457)
Hmm.

Joe Leo (25:00.53)
Well, I'm always interested in the business side, Or maybe not obviously, but because I run a business. And I mean, to me, that sounded like, you that sounds compelling. Okay, well, I'm going to start building AI into my legacy Rails app. And I'm a medium to enterprise business. want the guy who...

wrote the library to come and help us. But then you mentioned something about different ways of monetizing. I think that's actually probably because I've run a consultancy for 11 years. I'm like, that sounds a lot better. Do a product, do something where people are going to pay you a few cents per transaction or a few hundred dollars per month or something like that. Yeah, well, it's interesting because Brian Helmkamp just parachuted out.

Justin Bowen (25:45.688)
heard both sides.

Joe Leo (25:52.977)
Co-Climate and then there was another gentleman also just wrapped up product and they both started consultancies and I'm sitting here like running consultancy for 10 years. I'm like, I would never do that. I would totally start a product company. So yeah, you know, the grass is always greener.

Justin Bowen (26:06.35)
Grass is always greener. jinx.

Valentino Stoll (26:08.028)
You know, what it sounds like is we're about to see a super critical moment here where Active Agent starts to create like a baby AGI style business support mechanism where it can identify the critical aspects of your business to focus on, you know, and then sell. And then if it doesn't sell, be like, this isn't worth it.

Justin Bowen (26:19.086)
I would love it.

Joe Leo (26:20.775)
Now you're talking.

Joe Leo (26:26.406)
Yeah.

Joe Leo (26:32.238)
Yeah, you could be like the SBA, like the Small Business Association of the AI world.

Valentino Stoll (26:34.724)
Right. that would be awesome.

Justin Bowen (26:39.66)
Yeah, and it's, makes me think about what we were talking about before is like marketing for, open source and marketing for dev tools as a challenge. One of the initial bits of feedback we got, on, and I'm doing this with my partner, Rhiannon Payne, who's also marketing director at Ruby central. And, one of the bits of feedback was it sounds like something we have to pay for.

just because of the marketing copy that you put on the website. And people were very vocal about not wanting to pay for anything. Like developers want everything for free. Open source should be free, right? And like there are in my mind, like two like prototypical examples of where you could go with open source to monetize. And that's like the pro enterprise gem side of things.

Mike Perham did with Sidekick or there's services like Honey Badger. Yeah, I know Krafka. I got to meet Magic, can't say his name properly, so I just call him Magic. He said that's okay. He's the creator of Krafka, another active jobs runtime for Kafka. And he has pro and enterprise licensing now. One of his bits of advice was don't wait 10 years to monetize it.

Joe Leo (27:39.334)
Or the mutant gem similar free for open source

Justin Bowen (28:07.022)
I was like, okay, that's good. I can't. But yeah, I think the other side of it is building out a service like what Honey Badger did for error monitoring. I think I started using Honey Badger maybe like 2012, 2013, like early on. Same with Sidekick actually. The startup I was working for at the time was one of the first pro customers. And what they've done, like they focused on Rails first.

Joe Leo (28:08.08)
Good advice.

Justin Bowen (28:36.674)
But now they do other languages, other frameworks, and that's kind of what I'm thinking. Like I don't want to lock people into using active agent or Ruby and Rails, like, but initially that's where I am and what I'm thinking about. So I'll start there. And if it makes sense to open it to other languages and frameworks, like it is model agnostic, but maybe there could be more there.

Valentino Stoll (29:04.454)
Yeah, that's an interesting topic of discussion. It's funny how coupled what you build is to the model that you've chosen. And I'm curious your thoughts if you even agree with that one. also, like...

Do you like, do you look to, cause you have a lot of machine learning like background. And so like model selection is a very important part of like machine learning, right? So like, do you think about that in the AI space or is it just like pick pick one and go with it? You know, like.

Justin Bowen (29:41.059)
Yeah, no, definitely. Like you need to know not just the model, but all of the parameters he used and like generate like, and this isn't built into the framework now and testing and evaluations on your own terms, but I would highly suggest, generating like a, checksum value, like a fingerprint, like you see in the API responses of some of these providers of whatever it is you put into it and storing that in a database.

or in active storage or somewhere for reference. So you know it was this model, this version, these parameters that generated this result. And that's for testing and for production. It's not just about like the cost and like number of tokens, but it's like, did changing the temperature do? What did this prompt as part of that fingerprint do? And the fingerprint will change as you change all of those things. And that's something I've built out in Python for computer vision.

Valentino Stoll (30:21.479)
Yeah.

Joe Leo (30:27.694)
you

Justin Bowen (30:39.362)
And that's why I was like, I'll build a computer vision application framework because I've had to redo a bunch of stuff in open CV every time. And first time I did it with celery. If you're familiar with celery, the, And then the second time I did it with factory, which was fun. If you're familiar with factory, it's a go service built by Mike Perham to be language, agnostic queuing and de queuing. So it was fun.

Valentino Stoll (31:02.544)
Yeah, you know, that was one. This reminds me of like when I saw this library was like rail specific. just like my mind started going everywhere with like all the different tooling that's available in rails, right? Like and that like you've been building into, you know, looking in like you have like active support notifications, right? For instrumentation, which is like awesome, like because there's if you if you're like doing instrumentation and rails, you're like already hooking into that, right? And so like.

It would be like a one line change to add active agents instrumentation like to that pipelining, which is awesome. But like I start to think about other things too, right? Like, you know, like action mailer previews, like what does an agent look like previewed? Like, you know, what does a view even look like as an agent? Like I'm curious, like, what are your thoughts? Let's focus there. Like, what do you even like imagine an agent looks like from like a view layer?

Like it seems like there's so many permutations of that, which Reels honestly is a great candidate for with you where you could have multiple different formatting versions of a view for whatever you're trying to render. What does an agent look like in different formatting?

Justin Bowen (32:16.446)
Yeah, comparing it to Action Mailer is something I do a lot because it's very similar. I spent a lot of time thinking of, I didn't start off thinking of Controller. I'm pretty sure I probably started off where everyone else did with, it might be a model. But it's important to

just remember that it is an abstract controller. So any public method you define in your agent class becomes an action. abstract controller makes it so these action methods can be performed and have callbacks. And I've built in a queued generation similar to queued delivery for action mailer.

And one of the things I didn't do yet was the previews. Actually, I'm glad you mentioned that. It's something I've been thinking about as I am developing and testing. I think the simplest way to think about views for an agent would just be the prompt template that you're using or instruction templates. So like a plain text email template is something you could consider it like.

analogous with. But then from the example of using Next.js as part of my inspiration, generative UI is the concept of being able to render view partials like HTML partials or JSON partials, depending on how you're trying to do it. And I think that's where things could be more interesting. And like, yes, in conversations, you can imagine like even just like the iMessage

interactions you can have like sending money in a conversation is something that iMessage brought up. Actions and interactions are actually something we've built into a client project in 2020. And I really hope they have gotten on the whole large language model thing because it was like an SMS interface to communicate with underserved communities and help them go through these pathways to becoming

Justin Bowen (34:33.454)
more successful and having more resources. And so you can think about it in a chat interface of like, yeah, when you share a link, it gives you a link preview, but it could also give you like some kind of interactive form builder experience. And that's what we were building when I was at Fountain, which I don't think is necessarily like the best solution. Again, like everyone's thinking, chat bots, like we'll just have chat do everything, but.

Joe Leo (35:00.823)
you

Justin Bowen (35:01.368)
When it comes down to it, like filling out a form is often faster and making someone fill out like a five field form through a conversation can be painful. Even if it's cool and has like a location lookup widget and like other widgets that you can interact with, it's still just a form in a conversation.

Joe Leo (35:12.93)
you

Valentino Stoll (35:18.992)
You know, I'm leaning more and more every day to just like single text box and have it do a bunch of stuff. A button click.

Justin Bowen (35:25.612)
Yeah, or you know, like a button click that does something. good example, I heard of recently, someone was using Active Agent just for translation. So that translate button, translate it to my locale. When you click translate, very simple, like context is small. It's simple. It's not this back and forth agent interaction and it's not a complex workflow, but it is a good use case. I think one of the challenges is again, like, I hate the example of like a weather app.

Joe Leo (35:36.502)
Yes. Cool.

Justin Bowen (35:55.555)
I've been trying to figure out like what should like the like basic example app be like that's it's challenging and it, it depends for whatever your business like is doing, you know, like another one I heard was, in quizzes to help instruct the student on their wrong answer on like what to do better. You could provide them with a hint.

Valentino Stoll (36:03.334)
Weather.

Joe Leo (36:09.569)
Yeah.

Justin Bowen (36:25.208)
button that provides them with a hint with the context of the question and how they answered wrong. And depending on what they're answering, like that could be interesting because the context might be different each time. so giving them some kind of, again, prompting to the user, to that student of like, you know, like not just being like, answered this, but it's actually this, you know, kind of guiding them based on their answer to the right question.

Joe Leo (36:30.848)
Mm-hmm.

Justin Bowen (36:55.267)
teacher.

Valentino Stoll (36:57.094)
Yeah, you know, know, context building is such a interesting concept, right? Because like that it's like almost as critical as the prompt that you use. I would say maybe more so in a lot of cases. But it's also like how you gather up all of that context is separate from like how you format it as well. And the point at which point do you gather the context is also important.

It's almost like, it almost seems like there's a context framework missing of like gathering up context as you're using and doing something to the point of use. Like, and it seems like they're two separate, completely separate things, right? Even from an agent, right? Like, cause we talk about like, even in the industry, like retrieval, like going out and fetching external information, right? Like at the point of use, right?

Justin Bowen (37:39.682)
Yeah.

Valentino Stoll (37:56.038)
But it's like, there's a whole bunch of stuff that happens before you even get there. That's also valuable, right? And like how you gather it and get it to that point of use is like just as important. So like, I'm curious what your thoughts are on like, you know, what, do you try and do to like make sure that you have the right context for those two different kinds of use cases, right?

Justin Bowen (38:19.67)
Yeah, I'm going to try to answer without saying the word context. But, and I'm glad you said retrieval and not retrieval augmented generation or rag, which I think has become synonymous with vector search for some reason. But we were, we were using vector search on Cal faces in 2016. was no large language generation, in the background. think.

Valentino Stoll (38:33.97)
trying my best not to say that word.

Justin Bowen (38:45.238)
What other frameworks I've seen, and I actually like you saying that it's like another framework because within active agent is like two separate frameworks. There's the generation provider framework and there's action prompt and action prompt is like that abstraction layer on top of abstract controller. The generation provider is that abstraction on top of large language model interactions, as well as having some form of.

I have to say context management. But in other frameworks, what I've seen, I know, close. In other frameworks, what I've seen them call it is memory. Joe, I don't know if you use like the long or short-term memory within Crew AI's framework. It seems like an interesting concept. Yeah, like, and then.

Joe Leo (39:19.552)
You didn't even get that far into the answer.

Joe Leo (39:32.832)
I know. I'd have to ask my engineers. Yeah, I haven't heard it come up, so I'm going to say no, but I could be wrong.

Justin Bowen (39:39.247)
I don't think Vercell really touches on it, but Maestra again, which is built on top of Vercell, it's by the people that did Gatsby. If you guys are familiar with the web framework Gatsby, yeah. They also have this concept of memory and it's something I considered like last summer working on as part of like an initial release. But then I was like, you know, I have to like do something. I can't just like do everything at first. But one thing that

Valentino Stoll (39:48.742)
Mm-hmm.

Justin Bowen (40:08.556)
I'm glad I landed on active agents as controllers and like the naming active is a tough one to an action. Actually, DHH posted about active job and the new continuation feature for resumable jobs. And he posted action job. So it made me feel better about, cause like active agent talks to a background action mailer talks to a background.

You really shouldn't be doing these things in the foreground, but it is a controller. And the aspect of it being a controller gives you a lot of opportunity to take a step back and think about it like an MVC framework. And in my controller actions, what models do I need to query for? What third party services might be needed within these actions? And what's the best way of proxying those into my, you know, Rails app.

And just thinking about it as Rails application actions that you can retrieve whatever data models that you need and then perform some kind of process on in the action. And then like that helps you break down these problems, which again, I think it's like a lot of people are trying to figure out like, what is long-term memory? Like, is that the whole conversation?

Joe Leo (41:29.012)
Right.

Justin Bowen (41:30.744)
Is conversation really like, is chat and messages the best way to think about these things? Like you could do it for like logs. You know, like a log, like paper trail of things could be your context. I said it again, but.

Valentino Stoll (41:46.278)
Yeah, it's funny. I feel like, you know, Obi's first Magma chat, you know, that he had a concept of memory where it was just like a log at first. It just like went through a timestamp, right? Like, I think he even had topics attached to him for like tagging and doing better retrieval. I'm glad you brought up memory because there's like some very open concepts of evaluations attached to it.

Justin Bowen (41:53.336)
Mm-hmm.

Justin Bowen (42:11.672)
Did it help you remember something?

Valentino Stoll (42:15.782)
to memory and retrieval. Right? Like there's precision and like recall and there's like known bounds to and trade offs for each like aspects of these, you know, trade offs. And so I'm curious, like if you have tooling or, or like how you reason about like those specific aspects, right? Like of

of weighing the trade-offs for what is going to be most valuable for this particular retrieval mechanism or storage feature, right? Like as it relates to like the AI's usage.

Justin Bowen (42:58.146)
Yeah, I think again, it's going to be a challenge without the context of what product you're building, but it is really going to end up being similar to just best practices for building a controller or for building background jobs, knowing that like, again, an action mailer action is going to be performed in a background job most likely. And background jobs should kind of have a focused unit of work. And if you need to do anything else.

Valentino Stoll (43:26.354)
Mm-hmm.

Justin Bowen (43:27.438)
You can always have another background job for that. so keeping things concise is very important. That's actually one thing I really liked about, um, Oby's, uh, talk that he did last year. And I think he did something similar at a Railsworld, uh, as well, just on the topic of his book and like these small workers, uh, as he called it, something like that. And like having your agents again.

be focused on some kind of resource or objective or operation, some domain that like, this is a travel agent. It helps you plan travel. Like that's a concept I've been thinking about toying around with for personal reasons and past experience, but

Valentino Stoll (44:01.5)
Yeah.

Joe Leo (44:05.725)
Mm-hmm.

Valentino Stoll (44:09.456)
You know, it's so it's so easy to let you know, to not close the loop on a lot of things, right? You mentioned like, just throw things in the background. And it's like, if you've ever played with any of these code generation things, like it's so easily like get stuck in a loop of its own of just continuously generating the wrong thing that are not wrong thing, but like it doesn't know. But like generating the thing that you don't desire. And so yeah, I mean, closing that loop is like

Joe Leo (44:26.972)
us.

Joe Leo (44:31.741)
Mm-hmm.

Justin Bowen (44:31.949)
Yeah.

Valentino Stoll (44:37.658)
I feel like one of the biggest challenges of making agent-related things. What are your best practices or guidance on that, keeping things tight and consolidated to the goal?

Justin Bowen (44:53.166)
Still working on best practices. Docs actually have a lot of love coming very soon. I have something in a branch that I'm gonna publish, but I think best practices are still something that I'm working to define. Again, that's why like contracting and having more projects to help build out the framework is something that's important and I'm looking for.

Contributors, both individual developers having issues using it and companies trying to build products into it. but I think from that context, I said it again, from that perspective, it's kind of similar to like retry logic for background jobs, right? You don't want it to just endlessly retry. Like at a certain point, it should just be a dead job. And so it's similar in the sense of like this agent should have.

Valentino Stoll (45:30.546)
You

Joe Leo (45:33.436)
There you go.

Justin Bowen (45:47.797)
some opportunity to perform, it actually might need retries because of a network issue or rate limiting or something like that. But other than just basic retry logic, it's like, give it a certain amount of cycles or runs or steps. I've heard different frameworks use different terms. That's another thing, like request response cycle is something that we've gotten very used to.

Joe Leo (45:55.121)
Mm-hmm.

Valentino Stoll (46:12.466)
Yeah.

Justin Bowen (46:16.6)
like the prompt generation cycle is something that I've been playing around with as like a concept. And then like, yeah, agent to agent interactions. You know, we've been doing stuff like that. Servers make web requests to servers all the time. I actually joke about this in like a facetious way that everyone's an AI engineer now because we all make web requests. Right? So yeah.

Valentino Stoll (46:20.816)
Hmm.

Joe Leo (46:22.107)
Hmm.

Joe Leo (46:38.606)
Yeah.

Valentino Stoll (46:41.137)
Thank

Justin Bowen (46:43.31)
I think it's, these are definitely very interesting questions and similar happened on the last podcast I was on. It's like, this is great. and like, again, like working with clients has been great inspiration for figuring out like, what else can I build into like this free open source? And then what are like the valuable things that people are willing to pay for? That's something I'm still figuring out.

Joe Leo (46:43.48)
And,

Joe Leo (47:10.585)
Well, you know, this is interesting to me, because I clocked this gem before the show started, and it's like, just looking at the lib directory, 1,450 lines of code, 1,444 of which are Ruby, and it's just, it's tight, it's concise and expressive, and I love it. And then I'm thinking, man, what's he gonna do this thing? Like, what's he gonna add, right? How big does it get? Because when it's this small,

First of all, I didn't think that you could, I expected the library to be larger just to do what it does today, which is substantial. Rails does a lot for us, which is excellent and reassuring to know. But you have something that is this small, it's easy to test, it's easy to manage. But I'm curious to know like, okay, when inevitably either new requests come in or new use cases come in.

Justin Bowen (47:44.898)
Rails does a lot for us.

Joe Leo (48:05.647)
whether or not those are tied to business incentives. What happens to the library? Does the library itself grow or do you imagine it more in like a universe of libraries that can be built?

Justin Bowen (48:20.044)
Yeah, I've thought about it in regard to bullet train or jumpstart rails, if you're familiar, or just rails itself. Rails is made up of many gems. And like I said, action prompt could be a separate gem. The generation provider gem could be a separate gem. There are other gems that are very similar to the generation provider gem. what I'm...

Joe Leo (48:26.982)
Hmm

Justin Bowen (48:44.462)
planning on and haven't really done yet is breaking those into separate gems, but they'll all live in the same repo similar to Rails. And you could install them separately or together with active agent as like that core gem.

Valentino Stoll (48:57.328)
Yeah, you know my my gold standard of that is Samuel Williams. I just love his his structure. It's just so tiny if you ever go look at like Any of his like async related things they're just like hyper focused super small and then like they'll just reference a bunch of other gems if you want them right like you want to add like, you know streaming HTTP there's a C HTTP just for that and it makes it both

Justin Bowen (49:02.616)
Yeah.

Justin Bowen (49:12.75)
I love the async-related things.

Joe Leo (49:17.243)
.

Valentino Stoll (49:26.086)
harder to work with, but also easier to work with from a reading perspective.

Joe Leo (49:32.219)
you

Justin Bowen (49:32.845)
Yeah, even individual generation providers. I've thought about like, maybe I shouldn't have these be within the core gem. I've noticed like other libraries, like for cells, AI SDK, they're all separate packages for each provider. And that opens it up to like, and Mike kind of did something like this with factory where like, I think he supports Ruby and go, consumer and producers. Yeah. Yeah. But like the Python one is a community.

Valentino Stoll (49:39.772)
Yeah.

Valentino Stoll (49:56.304)
Yeah, the F A K T or what? Yeah, that's super cool library. Yeah.

Justin Bowen (50:02.878)
package and it opens it up more to having community maintainers of these things that I might not want to do. and like might not need for any of my projects. So like it, it's, it's something I was talking to Chris Oliver about too, is, his pay gem. He supported a different pay provider other than Stripe and he regrets it because now he has to maintain it even though he doesn't use it. And it's like, that's something I'm.

Joe Leo (50:16.269)
you

Valentino Stoll (50:28.902)
Right.

Justin Bowen (50:31.928)
trying to be mindful of. Yeah, we'll see.

Joe Leo (50:35.767)
Well, this is actually, this dovetails into what I was thinking about the long game of gem maintenance. You you mentioned Mike Tolesio at the top of the show. He's one of my, one of my best friends in this industry. And he has been maintaining Ruby gems for like almost as long as I've even been writing Ruby or probably longer than I've been writing Ruby. He's Nocogiri. Yeah. But, but many others, you know, I know. Yes. Yes. I'll appreciate that. You know.

Justin Bowen (50:42.658)
Yeah.

Valentino Stoll (50:55.194)
Is he the Nokogeuri guy? Yeah. Wow, just like, hold on.

Justin Bowen (50:55.414)
Yeah. He is the no go gear guy.

Justin Bowen (51:02.24)
Yeah, I know, right?

Joe Leo (51:05.593)
But many others, you know, I think I was looking at I think I was looking at Tailwind CSS the other day and you know, I want to look at like the latest release or something like that and you know, and it's It's you know, it's Mike D'Alessio who's you know, just done the latest release I'm like, you know, I texted him but you know, I think having released a gem myself in the distant past like I think you think that you're gonna release this thing people are gonna use it or not and

community will maintain it, but really it falls on the shoulders of one person. Not always the creator, know, Mike didn't write No Kagiri, but he's the maintainer. So it falls on, know, exactly, exactly, right. And so I guess what do you think about that? Well, I've got a two part question. One, what kind of collaboration and also requests are coming in now?

Justin Bowen (51:45.388)
Yeah, Nate Berkopec didn't write Puma, but he's now the current maintainer.

Joe Leo (51:58.999)
this gem? you know, it's early, but it's still, you know, popular for its age, I'll say. And then also, what do you think long term in, you know, in terms of growth and community support? Or hopefully just, you know, just in support.

Justin Bowen (52:11.704)
Yeah, I mean...

Yeah, well, I definitely, that's actually one of the questions I had, the last time SF Ruby was at Chime, think was like, and this was before I even released, like published any gem for it. I just had like the website and they were like, do you offer long-term support? And I was like, how do you know you want long-term support? You, no one's used it yet. And it is something that I am.

Valentino Stoll (52:33.81)
You

Joe Leo (52:37.549)
Ha ha ha!

Justin Bowen (52:44.216)
passionate about and currently all in on in terms of like, I've talked to a number of founders and people looking to contract me for Python work. And it's been tough to say no, because there's more interest in me doing Python and Next.js work than there is in doing like Rails AI contract work. And yeah, that's like the whole thing. And that was what my like,

Joe Leo (53:00.077)
Mm-hmm.

Joe Leo (53:07.959)
Well, we're here to change that.

Justin Bowen (53:13.484)
Rails 20, or sorry, RubyConf 2022 talk was about, was like, if we wanna do AI in Ruby, we can. I actually put together a proposal with the former interim executive director at Ruby Central. I presented a proposal for some kind of Ruby AI working group. Because again, this is something that like, it shouldn't just be on Andrew Kane to build everything.

Like the Onyx runtime Ruby is something I used just for fun to see if I could do inference on my GPU instead of using Python. And it didn't work. I opened an issue and he fixed it in a week. And now CUDA execution provider support works in Ruby. So you can do computer vision inference on your GPU and.

Valentino Stoll (53:55.74)
Just words.

Justin Bowen (54:08.878)
It's just as fast as Python. There's nothing special about Python other than the corporate and academic sponsorship that they've had. And that's, you know, there, there are a lot of tools we're still lacking. Like large language models aren't everything. I don't know if you guys are familiar with Jupyter notebooks or like Google CoLab. It's a way I've used to manage data scientists to get like their thoughts on paper so that engineers can actually build products out of what they're thinking.

Joe Leo (54:26.286)
yeah, absolutely.

Joe Leo (54:36.598)
Yeah, that's great because you actually can't build the product out of the notebook. you just cut them off right there. I know.

Justin Bowen (54:40.615)
Yeah, no data scientists shouldn't be building product exactly. Yeah. And

Valentino Stoll (54:40.903)
Yeah, you know.

It was a wild experience using Jupyter Notebooks with Ruby. I did a workshop with Landon and Max Erwin doing basically search in the context of AI and it was all with Jupyter Notebooks and Ruby stepping through each thing. It was pretty wild. It's hard because...

Justin Bowen (54:48.332)
Yeah, there is like iRuby notebooks. Yeah.

Joe Leo (54:49.685)
yeah, Landon had that talk on it,

Valentino Stoll (55:11.62)
As Rubyus we're so used to the IRB experience where you just jump in and you can just do anything you want. And Jupyter knows it's very like, step, here's a line, execute it and then it'll give you the result, right?

Justin Bowen (55:24.706)
mean, it's kind of like an interactive console. Like each cell is either marked down or code. And then like a code block can be executed and you get the results.

Valentino Stoll (55:26.844)
Kind of the same.

Joe Leo (55:31.457)
Mm.

Valentino Stoll (55:34.866)
But as result too, if you jump down the list, it's hard to get used to that, right? Like if you jump down the file and you're like, I'm going to execute this and see what it does. It's like, oh, you have to execute everything before it manually, right? And so like that was hard to get used to. But I do love just being able to like execute it and see it and like nicely formatted and you can provide all the data sets that you need. Really awesome.

Justin Bowen (55:41.592)
Yeah, you need to have it.

Justin Bowen (56:01.772)
Yeah, and you can run it in a Dockerized container environment. That's another thing I've been considering as like a business opportunity, similar to hugging face spaces, having these interactive runtimes where you can have your own Docker image because like Andrew Kane actually has a Docker image for CUDA support on the Onyx runtime Ruby to get like, you know, a proper version of CUDA installed so that you can use the Onyx runtime with CUDA.

Valentino Stoll (56:23.296)
cool.

Justin Bowen (56:29.9)
even though it wasn't working with Onyx Runtime Ruby, he had all of that and that's stuff I had to do for all of the computer vision projects in Python as well. sharing those kinds of dependencies, sharing these snippets of code and example applications, think is really lacking in the Ruby community. Elixir has live books. I don't know if you guys are familiar. It's basically like, yeah.

Valentino Stoll (56:55.588)
I love live books.

Joe Leo (56:57.448)
No, I haven't used them.

Justin Bowen (56:59.02)
And so like, he's great. Yeah. They have a lot of first party. Yeah. They have a lot of first party support for cool things like that and other machine learning libraries because of him. And I think I'm really excited. Have you guys seen what the evil Martians are doing with the rails guides to make it interactive and in the browser, like running in the browser? Cause there's a lot of cool things like Docker environments are great, but having access to like a

Valentino Stoll (56:59.174)
Jose Valim, he is magic. He's really missed, I think.

Joe Leo (57:01.782)
Yeah.

Valentino Stoll (57:12.178)
Have you played with the...

Joe Leo (57:17.355)
Mm-hmm.

Valentino Stoll (57:19.3)
So cool.

Justin Bowen (57:28.706)
browser you can just open and run and use your own computer's resources, but not have to worry about installing stuff. Or at least having a simple way of installing it in some kind of container environment, I think is really cool.

Joe Leo (57:39.766)
you

Valentino Stoll (57:40.696)
It is remarkable to how much you can run on a web GPU even.

Justin Bowen (57:45.922)
Yeah, WebGPU is very impressive. I'm excited for more local model runtimes.

Valentino Stoll (57:50.416)
Yeah, totally. mean, Hugging Face is just absolutely crushing it in this space, I think. I've played with a lot of their JavaScript tooling, where you can run ONIX models in the browser with web components and stuff. And it just works. You just say, hey, use this model and run it. It's slow, but that's just now. All of those problems will get solved over time.

Joe Leo (58:11.274)
you

Justin Bowen (58:14.424)
They used to actually have better free GPU Docker containers. But yeah, you can actually, I think a lot of Rubyists don't recognize, you can still download those Onyx models and run them in Ruby. I met a Rubyist at RubyKaigi that was doing exactly that for vector embeddings using the Onyx runtime. No large language models to do image-based vector search. So again, like.

Valentino Stoll (58:17.585)
I know.

Valentino Stoll (58:41.148)
Yeah, you know, if you guys want to learn more about that, at the Ruby AI Builders Discord, there's like discussions all the time of people like just running pure vectorization, like search locally super fast. Like you wouldn't believe some of the throughput and like people have been getting with Ruby vector generation and doing semantic similarity first just locally. It's pretty wild. mean, Mistral is surprisingly good at that as well.

getting those more dense vector stuff. I guess that's a question I have for you. When people are starting to look into semantic similarity related search mechanisms, coming back to this model selection aspect, what's your workflow for aligning the use case to the model that you're selecting? Because I feel like that's like a...

That's always a roadblocker to me personally when I'm like, I want to use an onyx model. And then I just see all the models there are. And I have this use case and it's like, just like, it becomes too much for me to sit down and think about that. just like move on. Like what's, what's your like thought process and workflow for like aligning that better.

Justin Bowen (59:50.723)
Yeah.

Justin Bowen (59:56.567)
It depends on the stage of the product and the infrastructure you already have to work with. Like I'm big fan of again, Andrew Kane. Like I think I'm the president of the Andrew Kane fan club at this point. So PG Vector and the Neighbors Gem, but you can use the Neighbors Gem with other vector embedding databases. And I think.

It goes back to having some kind of test bed for those things where you can be like, okay, here's an example of doing it with this, but like, here's my dataset and I need to see how that works in it. and yeah, it, it is a challenge right now to do a lot of these things in Ruby. Like I know Gusto is actually, has some Python developers at least, or, hiring Python developers as well to do like, yeah, right.

to do like model training and things where like, yeah, you could struggle with some of Andrew Kane's libraries to train models in Ruby instead of using PyTorch or whatever you might be using. But yeah, I think that it is kind of just like a trial and error and experiencing what you can and finding out what works. But again, if you already have Postgres, try it in Postgres. Like you don't need to.

spin up another dependency and...

Valentino Stoll (01:01:23.698)
So I'm curious what you see missing from the Ruby community in regards to like pushing people out, right? Like, how can we, what are we missing like to make this aspect like non-trivial, right? Like, or trivial, I always forget which one. Easier to like snap in.

Joe Leo (01:01:23.892)
Mm.

Justin Bowen (01:01:28.354)
I have a whole document.

Justin Bowen (01:01:43.497)
you know, it's not just tooling. think it is, it goes back to like the whole academic support. And that's one thing I'm really excited for Ruby Central's new executive director. she seems to have an interesting, fresh perspective of getting Ruby into schools. And I think that also will play into like a lot of this topic that we've heard for years now. I know Joe Masolotti did a talk at.

Valentino Stoll (01:01:50.768)
Hmm.

Justin Bowen (01:02:12.63)
Rails SAS that I got to see on the topic of junior developers and like everyone's going to be using JavaScript and Python because that's what they're teaching in schools like or Java and Python is what I had an opportunity to learn when I was sort of going to school and if we're not helping figure out what what is taught to new developers like we're going to miss out on a generation of new developers or

everything's just gonna be AI generated and... I dunno. Maybe.

Joe Leo (01:02:45.256)
Well, I'll be out of job, but we'll still have this podcast. So, you know, that's interesting because we spent the last, the first two episodes of the show, I think a lot of times we were saying, and I know I was saying, Hey, why not Ruby? You know, why not us? But one thing that I really love about active agent is that that's not asking that question. Active agent is saying,

Valentino Stoll (01:02:45.426)
Then it won't

Justin Bowen (01:02:50.008)
Something to talk about.

Joe Leo (01:03:09.756)
Hey, you should use this because look how well it leverages into Rails. And we've talked all about that this episode, right? Look at, look at how much Rails provides. That's why you should use it. There's a proactive reason, not just like a, Hey, you know, I like, I like Ruby and I like AI, so let me use it. And I do think that we are, we need more and Valentino mentioned tooling and you've mentioned sort of underwriting by

or others, you know, there needs to be more proactive like, hey, this is why you should choose Ruby over whether it's JavaScript, right? So what we need is next episode, we Riannon on here, talk about how we're going to market this thing. No, but in a real way, you know, it is the tooling, it is the saying, it is people saying, well, no, you should use Ruby because X.

Justin Bowen (01:03:43.752)
Better marketing. Yeah.

Justin Bowen (01:03:50.829)
Yeah.

Joe Leo (01:04:02.221)
I mean, and look, love Ruby. You said Ruby is your love language. I totally identify with that. I don't need a reason. But I think in general, like to the computing public, we should have good reasons.

Justin Bowen (01:04:13.548)
Yeah, you know, I heard you talking about Django on the last episode and I've always avoided it. I used Flask for the first computer vision application I built and fast API for the most recent one. And it's, it always just felt weird to me. And I just feel like Rails has a great opportunity to, kind of take the lead again in terms of like a framework of choice for building AI applications.

Where right now I would say Next.js really has like, I wouldn't even say use Python, like use Next.js. It's like full stack, but you know, use active agents. but like really like I would like to see more new startups use Rails as like their framework of choice, that one person framework to build an AI application. I would like more incumbents like the Gusto or Shopify.

Joe Leo (01:04:54.067)
You

Joe Leo (01:05:06.321)
you

Justin Bowen (01:05:11.96)
to continue using Ruby for more machine learning and other aspects, not just generative AI.

Joe Leo (01:05:19.698)
Good news there is that there are more more startups, especially from like greenfield startups that are saying, yeah, Rails, of course, that makes sense. At least they're saying that to me. Maybe I'm a little heavy handed. They're not gonna tell me they want to write in Python. But on a serious note, think there is an uptick in people saying, yes, I want to build with Rails. And of course, every business idea here has some kind of AI component.

Valentino Stoll (01:05:48.55)
Yeah, you make a great point about the academic aspects, right? Because even that's definitely one thing I've seen that has prevented the Ruby infiltration into all this machine learning aspects is the academic supporters of the institutions that are pumping all this stuff out, right? At Stanford or whatever, whatever team is there, they're using Python because they're supported.

like by Google or whoever else may be building these, or like Meta, right? To use the tooling and languages that they use, right? To build the byproducts. R, yeah. Or R, yeah.

Justin Bowen (01:06:25.546)
Or R. N Math Lab. I don't know. Have you ever seen a data scientist use Ruby though?

Joe Leo (01:06:28.178)
Yeah.

Valentino Stoll (01:06:34.394)
I have. So like that's what I'm saying though. I feel like, you know, the academic aspects, even from like a AI scientist, I guess we're calling them now, right? Somebody's like deep in the machine learning realm, right? Like our training models or all of these things, like, you know, all this tooling that has already been built, right? They're taking advantage of. So we have like,

Joe Leo (01:06:35.46)
I have, but they really like R.

Valentino Stoll (01:07:02.714)
Unfortunately a lot of catch-up to do in the Ruby community to build that tooling, right? And I think that is one thing that's holding us back even from the academic aspects is because like we now have to build that tooling that already exists where people are already saying well just use it and we'll learn how to do it, right? And so it is difficult there.

Justin Bowen (01:07:22.914)
Yeah, and that's why like the other side of it is not just academic but corporate sponsorship. So I'll share this proposal with you, but the idea is having like an open grant program where people could have some kind of proposal and having these like requests for proposals that companies are interested in sponsoring. Like having a direct method of sponsoring the development of a new gem to do something.

Valentino Stoll (01:07:35.419)
Okay.

Justin Bowen (01:07:51.618)
that is desired by a corporate entity with money and interest in Ruby.

Joe Leo (01:07:57.794)
I like that idea.

Justin Bowen (01:07:59.115)
I know that Marty Hott is doing a lot in that sense for just Ruby open source, leading the open source team at Ruby Central. yeah, I would love to see more financial incentives and investment going to the Ruby community for these types of tools and active agents.

Joe Leo (01:08:05.734)
Yeah.

Valentino Stoll (01:08:19.398)
Yeah, you know, it's interesting, like, thinking about, like the business aspect, right? Like, that's why people love Rails is it's like, it has all the utility and tools to just like make building business things like easier, right? To maintain, to create right off the bat. And I think that's why like, you know, YC is a huge contributor and like, promoter of Rails.

Joe Leo (01:08:21.699)
and active agents.

Valentino Stoll (01:08:48.08)
because of that, right? It's because you just make things quick. It's been a great prototyping, but also like long-term maintenance has been something that has been seen as like just also like straightforward path to success. And so I feel like that's kind of where we see this disconnect of the academics even is like we have like rails pushing as like a business thing. Yes, use this. Like it's going to be great and people are adopting it. But like then there's all these people from academics that like have missed.

that aspect of things. so like, where do you think like is that bridged gap? Grants are a great like starting point, but like, how else can we like show the incentives, right? To people to say, okay, like it's worth building this tooling because like really the business engine, which is Rails is like there and people are using it. Like it makes sense to marry the...

Justin Bowen (01:09:26.69)
and startups.

Justin Bowen (01:09:39.597)
Yeah, I think from running a business perspective, not having to hire a team of Python developers, like you already have a team of Ruby developers, enable them to do more with that. It's interesting you mentioned YC because I keep mentioning Maestra or Mastra. I don't know how they pronounce it. It's M-A-S-T-R-A.A-I.

They were part of YC Winter 2024 and they are building on top of Vercelles AI SDK, doing a lot of similar things to what I'm doing. And then I think they're doing like a hosted runtime for your agents as their business model, but it's like sign up and find out. But would YC accept active agents as like cohort member?

That's the kind of thing that we need more of is investment into Ruby shops doing dev tooling as like a way to build out Ruby infrastructure for machine learning and AI.

Valentino Stoll (01:10:39.986)
Yes, we are here.

Joe Leo (01:10:40.079)
Well, there's certainly a business case for these tools and I think not even, like yes, there's an investment case for somebody like for Y Combinator and Y Combinator you should invest in Justin Bowens and an active agents. But there's also, but I guess it shouldn't have to stop there. I I run a small dev shop, I've run a small dev shop for 10 years.

Justin Bowen (01:10:52.92)
Thank you. There's an endorsement right there.

Joe Leo (01:11:06.005)
One thing that we're really bad at is putting people to work on things that are useful and can actually generate income when they're not on a project, working on something. And we all have this. We all have this Slack or Beach or Bench, whatever you want to call it. it's a hard problem to solve. Some have done it really well. Most don't.

The fact is that there's income that can be used for these kinds of investments. And it makes a lot of sense to have something, even if it was always open source. Thoughtbot came to the fore by sponsoring a bunch of open source, probably kind of just took over a bunch of open source tools that everybody loved, that were not necessarily being built up and maintained. And they started maintaining them. They didn't charge anything, but everybody got to know Thoughtbot that way.

Justin Bowen (01:11:55.993)
Yeah, no, I mean, it's great marketing for consulting and that's why I'm offering consulting services. mean, that's what Evil Martians did the same thing. They built a lot of open source. They blog a lot about open source, but I think, yeah, there is a big challenge just industry wide, not only Ruby, but open source in general, that all of these large corporations, governments depend on open source and no one wants to pay for it.

Joe Leo (01:12:01.007)
That's why you're doing it. No, I'm not trying to pull back the curtain. Right.

Valentino Stoll (01:12:02.426)
You

Justin Bowen (01:12:24.832)
And like I had an idea on a beach, you mentioned beach in 2021 about like just starting a company that acts as that nonprofit for corporations to donate money to, and then have that be distributed through some kind of grant program system. And again, that's what I'm hoping Ruby Central might be able to do. But like, again, it could, that could be a startup in general across open sources, you know, GitHub sponsors is one thing.

Joe Leo (01:12:28.111)
Hmm.

Joe Leo (01:12:44.416)
Yeah, I think I think that might be in that room out. Yeah

Justin Bowen (01:12:53.196)
I don't know if those count as donations, so it's less incentivized as like a tax write-off for corporations. But you know, like that's not, I don't know how many people are like paying the bills off of GitHub sponsorships. I don't even have GitHub sponsorships active, should I? Like I don't know, yeah. I guess. That residual income.

Joe Leo (01:12:58.947)
I'd be surprised if they do, yeah.

Joe Leo (01:13:12.047)
I don't know. It's worth a shot.

Joe Leo (01:13:19.342)
Yeah.

Valentino Stoll (01:13:20.678)
I feel like that's every developer's dream. You know, we joked before the show, feel like, you know, being open source only developers is like a dream. Like the dream come true, I think for so many that just never comes to fruition. Right. Which is, which is fine. I mean, I feel like we still get the joy out of what we build no matter what.

Justin Bowen (01:13:23.288)
passive income for open source.

Joe Leo (01:13:24.909)
Yeah.

Joe Leo (01:13:28.366)
Yeah.

Justin Bowen (01:13:28.717)
Yeah.

Joe Leo (01:13:41.654)
Yeah, it's true. But it is kind of funny to think, well, yeah, right now I spend all day writing code in a dark room. But what I really want to do when I retire is spend all day writing code in a dark room for free. And that's the dream.

Valentino Stoll (01:13:51.578)
Right.

Justin Bowen (01:13:57.185)
I feel you.

Valentino Stoll (01:14:03.27)
Well, I feel like we've covered so many topics here. It's been great having you on, Justin. Are there any final topics that you wanted to discuss here? I mean, I feel like we could talk all day, to be honest.

Justin Bowen (01:14:19.776)
Yeah, I I mentioned it before we went live, but I am going to be live later this evening in San Francisco at SF Ruby. I think they're also trying to connect with the Chicago Ruby Meetup that's also at a Chime headquarter office in Chicago, which will be interesting at like the beginning, but it'll be live streamed and I will be doing a demo. It's simple and silly, but something fun to show what.

you can do with an agent oriented controller class. then yeah, contributors looking for people to either contribute, whether it's financially or code or issues, join the discord if you wanna participate in more conversations around the topic. It is not as poppin as the Ruby AI discord, but.

Joe Leo (01:14:53.336)
Thanks.

Valentino Stoll (01:14:53.426)
Honestly, is the gateway.

Joe Leo (01:14:57.09)
That's right.

Justin Bowen (01:15:16.342)
It's growing. think Valentino, you just joined, so you see. And yeah, do it. Open issues, complain about things. Docs will be coming online at least by next week. I'm hoping to do it tomorrow. Rhiannon said, don't try to do it today. She's also cutting me off. I can't keep working on my live demo after one.

Valentino Stoll (01:15:19.174)
Yep, I'm going to be more active in there as well as I start to like, yeah, we're using it.

Joe Leo (01:15:23.595)
Yeah,

Justin Bowen (01:15:43.117)
I mean, she said like, don't have to listen to this advice, but like, I've watched people like in the back of like meetups or conferences, like still working on stuff. mean, I've, I've worked on my talks, but I haven't done a live. I I've worked on my talk, like on the podium, at rails conf in Portland, but yeah.

Joe Leo (01:15:43.213)
you

Valentino Stoll (01:15:51.122)
Still working on it. Yeah.

Joe Leo (01:15:51.762)
yeah, haven't we all? Yeah. I work to my talks,

Joe Leo (01:15:58.669)
Hmm

Valentino Stoll (01:16:01.82)
I made the mistake one time of working, I gave a talk on this project I called Podcast Buddy. It just like locally listens into the mic and like summarizes topics and suggests questions and stuff. And I gave a talk at one of the Ruby AI New York City meetups. Before it was artificial Ruby, I feel like. And I was working in the back on it, like trying to add new features.

I'm like, wait, I have to give a presentation on this like any second now. And I definitely broke something like and luckily I fixed it. like, yeah, note to self, don't do that again.

Justin Bowen (01:16:34.414)
Yeah.

Justin Bowen (01:16:42.67)
And no, this is my first time going not back to back, but morning to evening. So in like 11 hours, I'll be starting my talk. So at 7.50 Pacific. Thank you. Hopefully I'm still awake. Maybe I can take a nap.

Valentino Stoll (01:16:55.258)
Hey, good luck. Yeah.

Joe Leo (01:16:56.394)
Yeah, yeah, it's gonna be great.

Valentino Stoll (01:17:00.188)
Hahaha

Justin Bowen (01:17:04.718)
But yeah, I really appreciate you guys having me on. It's been a great conversation and a lot of interesting topics for the future.

Joe Leo (01:17:12.5)
Yeah, it's a lot of The feeling is mutual.

Valentino Stoll (01:17:13.658)
Yeah, totally. I mean, we'll have to have you back on again, you know, after you've given any number of talks, I'm sure, as you start building this out and people start using it because it is addictive, you know, like, you get the right abstractions in place and snap it on the right framework and, you know, it just becomes easy to use and makes you use it more, you know. So I'm pretty pumped.

Justin Bowen (01:17:22.254)
We'll see.

Joe Leo (01:17:23.936)
you

Justin Bowen (01:17:39.257)
Yeah, I'm excited to make more content with it now that I'm getting it in a more stable place. Cause like, it's just been a lot of like, you know, trying to make ends meet contracting, trying to build up the framework and it's starting to feel like I can actually start producing some content. And that'll be fun, fun to.

Joe Leo (01:17:52.395)
Hmm.

Joe Leo (01:18:00.652)
Excellent.

Valentino Stoll (01:18:01.234)
will say to support that idea, I've used Sublayer quite a lot because of Scott Warner making all his content and being like, hey, this is something you can do. And I'd be like, let me try that out. And I'm like, sure enough, hey, that's great. So it does work.

Justin Bowen (01:18:07.363)
Yeah.

Joe Leo (01:18:11.201)
Mm-hmm.

Justin Bowen (01:18:13.795)
Yeah.

Joe Leo (01:18:16.161)
Yeah.

Justin Bowen (01:18:16.3)
Yeah, I'm looking forward to it.

Valentino Stoll (01:18:18.098)
Awesome. Well, so at the end of the show, we like to just pick something that we've been super interested in. It doesn't have to be programming related, but it can be. And just that you want to share with people, it doesn't have to be anything if you don't want.

Justin Bowen (01:18:39.158)
you know, I should have, I should have thought about this because I did hear you guys mentioned that at the end of your first episode. you know, I, I've, I'm going to do it. there's a great Netflix anime called rising impact. is just about golf and like a little boy that likes to the ball far. Like that is his special power is driving far.

And then there are other people with special powers. Like one person has like, they always sink a putt because they see a shining road to the cup. I don't know. Like I just got back from a six week trip in Japan and I was like, I want to watch something weird. And it's like a Netflix series anime and it's weird. I don't know why that's what came up. You said it doesn't have to be programming related. Like I've had that on the background burned right through it.

Joe Leo (01:19:15.723)
Thanks.

Yeah.

You

Joe Leo (01:19:29.471)
No, this is good.

Valentino Stoll (01:19:35.132)
That's awesome.

Justin Bowen (01:19:35.608)
But how about you guys? Because I just brought up rising impact.

Joe Leo (01:19:41.355)
Well, I'm gonna plug the most interesting AI startup idea I heard this week at Tech Week that is simultaneously the thing that I thought was, I thought this was a really dumb idea when I heard it and I'm totally wrong. It's actually brilliant. It's called Osmo AI.

And their thing is, I'll put it in the chat here, their thing is to, they're giving computers a sense of smell, which is the silliest thing I had ever heard until I heard about what they're doing. And they're like, one of, now they're a lab, right? But one of the businesses they've started off the back of this is, you know, working with perfumeries and they can actually, like, they shrink the time for like creating these custom scents to like a couple of weeks where they used to take like 18 months.

Justin Bowen (01:20:13.719)
Justin Bowen (01:20:17.517)
Love it.

Joe Leo (01:20:35.292)
And the guy did a demo by creating a scent for just a person in the audience. Just like, okay, what do you want? And it was very cool. I just, thought it was very cool. So.

Justin Bowen (01:20:41.71)
That's awesome.

Valentino Stoll (01:20:45.084)
Wow.

That was something from the CES, I want to say like 12 years ago I saw, where they had like a scent machine you could plug into your computer and like, yeah, and like it would do a similar thing where you like go to website and push a button and it would connect to the device and like produce the smell for you. So if you were at like dominoes.com, it would like produce the pizza smell for you, you know? Like a smell of generally, yeah. Right.

Joe Leo (01:20:58.536)
Really?

Justin Bowen (01:21:08.046)
See.

Justin Bowen (01:21:12.142)
Like a smell generator, whereas the other one's a smeller. Yeah, when you mentioned that, I was thinking like, maybe to like detect diseases, because you've heard of like mice that are like dogs that can, yeah. But no, no, just smell perfume and hope you make a better perfume.

Joe Leo (01:21:12.306)
It's awesome. Yeah. Smell generator, yeah. Yeah.

Valentino Stoll (01:21:24.898)
yeah, the breath stuff.

Joe Leo (01:21:25.938)
yeah, right, that's right, See that? Well, you know, think perfume, maybe that just generates the dollars in the beginning, but.

Justin Bowen (01:21:34.319)
I mean, hey, mine was weird. I did think of one, I guess like SF Ruby conference, but also there's gonna be an SF Ruby AI hackathon July 19th that I'll be judging. I don't know if it's been announced yet, but I will be judging there. And yeah, it'll be great. I know Marco Roth is coming out after RailsConf to potentially participate or mentor. I'll also be there mentoring July.

Valentino Stoll (01:21:48.038)
cool.

Joe Leo (01:21:48.459)
okay.

Valentino Stoll (01:21:58.972)
That's awesome.

Joe Leo (01:22:00.292)
right, so it's that next weekend? Right after RailsConf, mean. Yeah.

Justin Bowen (01:22:04.046)
19th. I hope it's not next. Yeah. The next weekend after Rillsconf. Yeah. Yeah. So I'm really excited about that. Yeah. Just found out about that last week. So.

Valentino Stoll (01:22:08.809)
that's gonna be awesome.

Joe Leo (01:22:10.378)
you

Valentino Stoll (01:22:12.69)
You

Justin Bowen (01:22:15.476)
more relevant than rising in pay.

Joe Leo (01:22:17.738)
you

Valentino Stoll (01:22:17.906)
I'll share this whole, I'll probably release it more officially by the time this episode airs, but I've been working on this project called agentic. It's a Ruby gem. It's kind of just exploration, but it's a self-building agent framework where you just give it a goal in mind that you want to accomplish. And it goes and it assembles.

whatever agents that it needs to accomplish that and sets out a plan. It's basically a plan to execute thing that is self building and it generates tasks automatically and then tries and completes those tasks. It's like super just fun, experimental. like take it with a grain of salt, but like I'm just having a lot of fun with this style of, yeah. And so.

Joe Leo (01:22:51.946)
Mm-hmm.

Justin Bowen (01:23:00.588)
Yeah.

Justin Bowen (01:23:05.358)
Sounds awesome. Way more serious than what me and Joe brought up.

Joe Leo (01:23:09.45)
Yes, that's admittedly.

Valentino Stoll (01:23:14.124)
This was something I was working on all last night, because I was so excited. I finally got to a point where it's actually doing the things. So, but yeah, just an interesting idea. And it makes me think too of like, you know, closing out like Ruby's expressive nature, like, and ability to create its own language, right with DSLs we've talked about before. And like, it'd be interesting to see what it looks like to have an LLM be able to create its own language using Ruby.

Justin Bowen (01:23:20.173)
awesome.

Justin Bowen (01:23:35.34)
Yeah.

Valentino Stoll (01:23:43.418)
define all of the different keywords and things that it needs to create its own language and kind of just see what it makes, you know, like I'd be interested in that.

Joe Leo (01:23:46.473)
Yeah.

Justin Bowen (01:23:51.535)
Hopefully it's not like a Python data scientist and everything single letter attribute, like variable names.

Joe Leo (01:23:53.769)
I know.

Valentino Stoll (01:23:54.578)
Maybe it makes Python, you're right. It becomes a Python interpreter, right? Right, yeah.

Justin Bowen (01:24:03.542)
just letters for variables.

Not words. Why words?

Joe Leo (01:24:07.933)
And for anybody that's joined the last two shows live, this is now the second release announcement in two weeks for Valentino, because this podcast is actually just an elaborately disguised launch pad for Codename V. Exactly. Maybe it's getting you some accountability. You've got all these projects out there that are half finished, and you're like, you know what? We've got to get this on a schedule.

Justin Bowen (01:24:18.914)
Yeah

Valentino Stoll (01:24:23.026)
That's my personal experimentation.

Justin Bowen (01:24:25.486)
I mean, it's your platform, use it.

Valentino Stoll (01:24:28.188)
I know. It's true, you know, I...

Justin Bowen (01:24:34.925)
Yeah

Valentino Stoll (01:24:35.794)
I made this list a while ago on my GitHub, like just code name V and I just like made a list of all the things that I've like experimented with over the years. And it's just like, it keeps growing. And so I figure if I keep adding to it, like maybe I'll come back and like now that there's AI to automatically like make stuff, like I can accomplish stuff and so far so good, right? I'm not going to keep up this effort though.

Joe Leo (01:24:58.023)
Yeah, yeah, absolutely.

Justin Bowen (01:24:59.459)
Yeah.

Valentino Stoll (01:25:04.847)
Don't expect too much more releases every time, but you never know.

Justin Bowen (01:25:09.612)
actually just reminded me of a quote I saw on Twitter which was it was beautiful and nothing worked talking about what an LLM generated for them. Yeah I mean that's great.

Joe Leo (01:25:19.146)
I saw that. Yeah, I saw that. It was great. Yeah. Nothing worked, but it was gorgeous. Yeah, that was GPT-4-1. no, was us, or was it Anthropix? It was one of the newest releases, models. well, I guess it was quoting, they were quoting Kurt Vonnegut then.

Justin Bowen (01:25:35.338)
No, I think it was Kurt Vonnegut that said that.

Justin Bowen (01:25:40.43)
I'm just using it.

Justin Bowen (01:25:46.274)
But nothing hurt. Anyway.

Joe Leo (01:25:49.992)
You

Valentino Stoll (01:25:51.836)
Well, thanks again, Justin. Yeah, this was incredible. I can't wait to keep releasing these episodes because I feel like we just have more and more incredible conversations and you were, you know, a key part of that. So I appreciate you coming on and sharing your awesome framework, ActiveAgent, because, it's really coming together. I remember seeing the release with just the landing page too and being like, wow, I can't wait to use this thing, you know? So, yeah.

Justin Bowen (01:25:51.886)
Thank you guys again for having me on, yeah.

Joe Leo (01:26:09.318)
Yeah, thanks for everything.

Joe Leo (01:26:18.535)
Mm-hmm.

Justin Bowen (01:26:19.352)
Yeah, hopefully the docs help with adoption and yeah, get more issues and contributions and looking forward to seeing you guys, we'd be at RailsConf or RailsWorld or awesome.

Joe Leo (01:26:23.452)
Mm.

Valentino Stoll (01:26:29.744)
Yeah, first time in a long time I will actually be at RailsConf, so pretty excited. Yeah, and the last time, that's right.

Joe Leo (01:26:30.13)
We will be, yeah.

Joe Leo (01:26:34.542)
Mm-hmm. And the last time, yeah. Mm-hmm.

Justin Bowen (01:26:34.698)
And the last time. Yeah. Awesome. Well, I'll see you guys there and thanks again for having me.

Valentino Stoll (01:26:41.904)
Yeah, sounds great.

Joe Leo (01:26:43.1)
Thanks, take care Justin.

Valentino Stoll (01:26:48.626)
Alright, cool.