Algora | Algora TV

COSS Founder Interviews #21 - Joe Sciarrino, founder of Hydra

Algora

Watching now

00:00 Hey, Joe. Thanks so much for joining me.

00:05 Oh, thank you for having me.

00:07 All right. Let's kick off the interview.

00:08 So I'd like to hear some information about your personal background first and then how Hydra got started.

00:15 Yeah, absolutely.

00:17 So my co-founder and I met at Heroku.

00:20 So if you're unfamiliar with Heroku, it's a cloud platform as a service.

00:25 JD was there before I started and he was there after I left.

00:30 I went to a startup called Citus Data, which was a sharded Postgres that could scale out horizontally.

00:38 That company was acquired by Microsoft Azure. So I spent the last couple of years as a PM supporting Postgres primarily,

00:47 but all open source databases at Microsoft.

00:51 Lovely.

00:52 And how did you come to the decision to start Hydra with your co-founder?

00:56 Sure.

00:58 We participated in Y Combinator last year.

01:02 And one of the things that we noticed were that our batchmates were using data warehouses.

01:08 So think BigQuery, Redshift, Snowflake, others.

01:13 Coming from Microsoft,

01:15 I traditionally thought that companies

01:17 wouldn't adopt a data warehouse

01:18 until they were multiple terabytes,

01:21 perhaps even tens of terabytes in size.

01:25 When you think about what a data warehouse is for,

01:28 it's for aggregating many disparate sources of data

01:32 to a single common place.

01:34 And it's also about processing large amounts of data

01:38 as an organization grows.

01:41 Now, these companies in YC did not have huge amounts of data, but they did see a lot of value in consolidating from many of the different silos to that kind of single place to do proper reporting. We realized that startups today, ourselves included, use dozens and dozens of different software tools, collecting, you know, usage.

02:15 Our companies was a real problem. startups using enterprise data that are really geared towards an open source data warehouse

02:33 that's more accessible to developers, startups, and everyone who's really looking to get started

02:37 with proper flexible analytics at their company.

02:42 That's amazing.

02:43 So close to a personal problem, personal experience, and then, you know, immediately next to your

02:48 batchmates hearing about their needs and building accordingly, building open source.

02:53 So what was the first launch like, you know, announcing to the world, you're building Hydra

02:59 and then, you know, fast forward to today, how are things looking like?

03:02 Then, you know, fast forward to today, how are things looking like?

03:02 Yeah, absolutely.

03:04 Yeah, absolutely.

03:12 Having been in YC, Hacker News community is one that's really near and dear to our hearts.

03:21 You know, more than half of the GitHub stars that we have on our repo are just as a result of posting some of our release to Hacker News. Back in December, we shared how we added query parallelization

03:27 and vectorization of where clauses to Postgres on columnar storage. In doing so, we were actually

03:36 able to benchmark Hydra as the fastest Postgres for analytics in the entire market using the ClickBench benchmarks.

03:50 So that got a lot of attention to the project.

03:54 And yeah, it's been really great to see that engagement.

03:56 That's really badass.

04:01 And you already touched on sort of like how the go-to-market looks like in terms of having conversations with people, with organizations, with teams,

04:06 and prioritizing the backlog, setting the next milestone, if you'd like to share this.

04:11 Just curious how you're navigating all this as a founder.

04:15 Yeah, absolutely.

04:16 So beyond performance and speed, what matters to founders and developers is the features and the functionality

04:29 that enables them to build really great analytics around their product. Every company,

04:37 every vertical is just a little bit different. So one of the really beautiful things about Postgres

04:43 is it has a really wide breadth of many features, functionalities.

04:47 And if you don't see it there, you can actually create extensions to add capabilities beyond what's just in the core Postgres feature set.

05:06 moving forward is how do we deliver an increasing amount of features and functions to our end users so that they can really take the warehouse and evolve it to their specific needs.

05:13 So excited to kind of share in coming months what we'll be releasing. But again, we're building

05:21 kind of on that rich foundation of Postgres. I think that when you build products, you have to start from the right foundations.

05:29 One of those foundations is Postgres and Postgres is extensibility.

05:32 But the other piece is open source.

05:36 Open source really has the right values that we believe are necessary for a great warehouse, such as collaboration.

05:43 for a great warehouse, such as collaboration.

05:47 And sadly, these are kind of values that were maybe not as emphasized

05:49 in traditional enterprise data warehouses.

05:53 And these traditional data warehouses,

05:56 I mean, that extensibility you just mentioned,

05:58 they like it.

05:59 You would probably need to get in touch with support

06:01 and then, you know, good luck.

06:02 So that's a superpower.

06:03 Yes.

06:04 Hydro, right here, one of the good luck so that's a superpower yes hydro uh right

06:05 here one of the many and uh that's basically you said also the next you know milestone to have these

06:11 uh new features and extensions built by the community uh what it's what is it like for you

06:17 managing uh you know contributors uh growing this community and if there's any lesson learned so far you might like to share

06:26 or a mistake to avoid. Yeah. Yeah. Well, let me emphasize that open source

06:34 and being completely transparent with building in public is a real superpower for any project

06:42 that especially deals with data or infrastructure, kind of at that lower level.

06:47 We've had folks open up issues around, you know, they ran into this problem or that problem,

06:56 and that's allowed us to find things that we might have not found on our own before,

07:03 especially with kind of a smaller organization

07:06 that's moving quickly.

07:08 It's great to see community members

07:11 jumping in our Discord, letting us know,

07:14 hey, I tried this and it didn't work as expected,

07:16 or I tried that and it exceeded my expectations.

07:19 That's great signal to us

07:20 as the kind of primary workers on Hydra, but there can be a whole other concentric

07:28 circle of people that are giving us feedback. And, you know, this is a really fantastic part

07:35 of being open source. And a good time for the project, it sounds like. And so with this

07:41 increasing, you know, community feedback and with the pull from the market, this all comes with its own bandwidth requirements on the team.

07:49 So curious how you have navigated growing the team, funding the project, the company behind it.

07:57 If you'd like to briefly talk about this.

08:00 Yeah, absolutely.

08:26 Yeah, absolutely. About April of last year, we decided that we really needed to do the hard tech thing, which was, hey, can we take Postgres and make it as fast as Redshift or as Snowflake? Can we do that? And in doing that, it's a big project. It requires a lot of really great engineers that are working really hard.

08:31 So, you know, we raised our seed round post-YC.

08:38 And since then, we've been, you know, strategically adding some of the really best people around to work on transforming Postgres and extending it into something that can be just as fast

08:44 as these kind of more

08:45 legacy systems for analytics.

08:49 That sounds great.

08:51 If I could ask for someone that is not so versed with data warehouses, you could maybe

08:57 explain with an example.

08:58 I read how basically Hydra decouples Postgres query, compute, and storage and sits on top of it.

09:07 So if you could just kind of explain this for a little bit.

09:11 Yeah.

09:12 So one of the key things to remember about Hydra is that we take everything that's good

09:19 about Postgres and we just add to it.

09:22 So Hydra is not different than Postgres to the extent that we've never encountered a single Postgres connector or integration that didn't just work out of the box.

09:47 transactional capability. So that is still in the product, something that is lacking from a lot of traditional warehouses that are just purely focused on OLAP, online analytical processing only.

09:54 So with Hydra, you have a lot of flexibility to define, you know, okay, you know, I have a large fact table that I'd like to put into a columnar format.

10:06 And then I have a high rate of data ingestion and throughput that I'd like to continue

10:12 managing as in a row based heap tables within Postgres. So there's a bit of a fancy term here.

10:21 It's called table access method. That's how our columnar is implemented.

10:25 So within the same Hydra database,

10:27 you can choose what tables are row,

10:29 which ones are columnar.

10:30 It gives you a lot of flexibility

10:32 in defining the workload

10:34 and the data modeling in the database.

10:39 Nice, thanks.

10:39 And that sounds like then how Hydra

10:43 also helps you make better analytics experiences for over all this data because it's so easy.

10:49 Yeah, absolutely. You know, I think there's a really great principle that maybe folks don't say enough, which is, you know, keep it as simple as you can for as long as you can.

11:00 I think that at larger scales, it absolutely makes sense to isolate between a

11:06 transactional system and an analytical system. But certainly at smaller scales, it can make a

11:12 lot of sense to consolidate both your transactional and analytical in the same place to the extent

11:18 that which neither of these could be impacted. I think columnar allows analytics to scale a lot better

11:27 so that they're not as impacted by a transactional workload.

11:31 Thanks for these insights.

11:35 For your team, from a go-to-market standpoint,

11:38 what kind of channels do you mainly leverage?

11:41 Is there outbound going on with big enterprises?

11:44 Is it mostly inbound and through marketing? How did you navigate this?

11:49 Yeah, primarily

11:50 most of our customers today are from inbound.

11:56 So we have a free plan that folks

12:00 can get started. This is a perpetually free plan that just goes to sleep

12:04 after 24 hours. You can

12:05 continuously wake it up if you'd like to. It's a great way to get started on the cloud. Also,

12:11 because it's open source, well, you could use the Docker locally on your laptop. So you could,

12:19 you know, run Hydra there for simple feature and functionality testing.

12:26 run Hydra there for simple feature and functionality testing. Yeah, we do outbound from time to time, but we've been lucky to have a healthy amount of inbound of people that say,

12:31 hey, I've got a data warehouse today. I think it's way too expensive. And they're looking for

12:36 an open source option. Or on the flip side, we also have people that have never used a data

12:41 warehouse before. And Hydra is really their first foray into

12:45 into using a data warehouse people that are familiar with using postgres but haven't really

12:50 used a data warehouse before nice so it's uh simpler to use it's much cheaper free and you

12:57 know it's much more performant uh so i mean that's a deal breaker right here, it sounds like. That's lovely.

13:15 Could we cover a little how you approach monetization, you know, from a sustainability standpoint and what different forms it's taking today might take in the future?

13:18 Yeah, absolutely.

13:26 You know, I don't think that pricing is always a way to define a competitive advantage. However, I think that, you know, Redshift, Snowflake and others really left a wide area of the market open by, you know, having pricing that is much higher than what is really sustainable for early developers

13:46 and startups. So like I said, Hydra is free to start, but the prices are also quite fair as you

13:53 scale up to. So, you know, once you go beyond the free plan, then we charge for a cloud managed

14:02 service, right? So think you're paying for the ops,

14:05 you're paying for backups,

14:07 you're paying for HA, disaster recovery,

14:10 in-place upgrades so that there's no downtime

14:13 maintenance window or anything like that.

14:15 So those are the kinds of things that you pay for.

14:17 Again, people could take our open source

14:19 and run it for free

14:20 because we are starting from a principle

14:23 of making the data warehouse accessible.

14:26 But yes, we hope people will run our managed service and they'll pay us for that managed

14:32 service.

14:34 Thanks.

14:34 Thanks for elaborating on this.

14:36 And is ever support with customers something that is part of your model or operations or

14:43 the needs of them?

14:45 Yeah, support is included for free with our managed service.

14:49 So we have a ticketing system that's powered by our friends over at Fogbender.

14:55 And also we have free community support in our Discord as well.

15:00 We even have a special channel that's called Experts.

15:03 So if you have a really complex issue, you can post it in there for even, you know, deeper support.

15:11 That's really cool, actually.

15:13 Lovely, lovely.

15:14 And today, your day-to-day, what is it looking like?

15:18 To what extent do you write code and, you know, speak with users, with customers?

15:23 What's it looking like?

15:23 Yeah, absolutely.

15:24 So what I'm focused on is product and go-to-market.

15:30 This actually speaks a lot to my previous experience at Heroku and Citus and Microsoft,

15:36 because those are the things that I was doing then as well.

15:39 So it's great.

15:41 I get to do my favorite thing, which is talk about data and talk about Hydra with people that are

15:48 trying it out for the first time. So helping with

15:51 implementation, helping with account management, and also doing

15:55 great programs like this is a really fantastic

16:00 thing I get to do every once in a while.

16:03 I really appreciate the opportunity

16:05 and i mean i know how busy founders are so you know it's it's i really appreciate having you here

16:11 to share share the story um that's that's phenomenal that's what also you know a lot of

16:17 other founders lives uh looking like you know wearing all these hats and doing all these cool

16:21 things working product talking to customers um is there, have there been people in the open source ecosystem that through your journey,

16:29 you know, you've been in close touch with or shared advice to be, you know, some advice

16:34 of your own that we could share with people getting started?

16:37 Yeah.

16:39 Yeah.

16:41 You know, Hydra is built with community in mind.

16:47 As I said before, you know, we want Hydra to be adapted and evolved to your specific kind of industry specific or company specific needs.

16:59 So if there's something that you'd like to see in Hydra, you can contribute it yourself. You can open a pull request.

17:07 You can flag issues as they arise.

17:11 We'll certainly give it a lot of attention.

17:13 You have our focus.

17:15 If you make some noise in our repo, you have our focus.

17:20 And I believe we shared a template that people can use to contribute.

17:28 And if you're not finding what you need, please jump into our Discord because that's a great way to talk not just with myself, but the rest of the Hydra team directly.

17:39 That's phenomenal.

17:50 That's phenomenal. And now in terms of kind of like the next steps immediately, you already talked about, you know, a little bit how the roadmap is looking like and next milestone. Is there something we could say about, you know, until the end of the year?

17:53 Maybe what people can look forward to in addition to all these new extensions and this? Yeah.

18:00 Absolutely. So Hydra, because it's a database, we're very conservative with our releases.

18:10 Your data is very important. If your data is like gold to the company, we want to be good keepers of it together with you.

18:18 And so we're moving towards releasing Hydra 1.0.

18:23 We're moving towards releasing Hydra 1.0.

18:26 This will be our main production release where we officially have removed the Hydra beta tag.

18:30 That's coming soon.

18:33 And I think what you'll find is a really easy way

18:37 to use columnar storage right in Postgres.

18:40 We've made improvements, like I mentioned before,

18:42 with query parallelization on the performance end.

18:45 But then we've also added improvements around vacuum and space reclamation and all those kinds of things that are needed to actually operate a really efficient data warehouse and production.

18:57 It's a very exciting time.

19:00 Yes.

19:00 I'm curious to ask, how did you decide to use the name Hydra?

19:04 I should say it's a Greek, you know, mythical monster with many heads.

19:09 And so how did that. Yeah. How did you choose it?

19:12 Yeah, absolutely.

19:13 So we're looking at an architectural diagram and seeing many data sources all coming to a same kind of central place, a central body.

19:24 So to me, that really looked like a Hydra.

19:28 Also, I think it's just a kind of a cool name.

19:31 And I hope other people like it as much as we do.

19:35 It was difficult to get the domain name.

19:38 So, you know, you can find us at Hydra.so.

19:43 And also last week, I purchased OSSwarehouse.com.

19:48 I was actually surprised that no one had ever purchased that before.

19:51 And it wasn't a $15,000 domain.

19:54 It was like $6.

19:56 So OSSwarehouse.com will redirect to the Hydra website too.

20:01 That is neat.

20:02 And, you know, that's something that's another founder's blog post about, you know, getting into the main spoiler projects. That's right. That's really

20:09 cool. And according to mythology, if you slay, if you cut one of the heads of the Hydra, you know,

20:14 more growing in its place. So is there an analogy here we can draw? Oh, that's a good question.

20:22 You know, not other than the fact that I think Hydra can be very resilient. So, you know, not just from data sources, but, you know, workload types as well. Hydra can serve both OLTP, transactional head, it can also serve OLAP head, and maybe there's going to be more of those in the future, too. So, like I said, I think that simplicity is really important.

20:49 So if you know how to use Postgres, you like to use it, and you don't want to add maybe a copious amount of databases, Hydra is a really great way to get started with building analytics for your startup.

21:03 I love it. And it's really, really hard engineering,

21:06 but it's also, at the end of the day,

21:07 something very simple and it's clear

21:10 and you're great at mitigating everything

21:12 for someone like me to understand.

21:14 Yeah, thank you.

21:15 Yeah, you know, no one was really doing OLAP Postgres

21:18 and we figured we would be the ones

21:22 that were crazy enough to do it.

21:24 So maybe this is naive, but we've really made surprising progress already.

21:31 So if you want to help us, please be a contributor.

21:35 We're like more than happy to welcome any discussions or new thoughts on where we can take things next.

21:43 Amazing, amazing. And things we will build that will be available to all of the other users.

21:48 And so, you know, there's going to be some real impact here,

21:50 improving the product for everyone involved.

21:53 That's perfect.

21:55 If you were to sort of like, you know, go back a little and, you know,

22:00 if you could ask your future self for some piece of advice,

22:03 or if you could maybe ask another

22:05 founder about the more sensitive topic that's not covered like what would that be um yeah

22:12 i want to make sure i understand the question so if i was um getting advice from my future self

22:18 what advice would we what would what advice would i be looking for right if you rolled back time

22:23 yeah that's a good uh that's a good question.

22:27 Let me think for a second.

22:36 So I think that as companies grow,

22:41 there's increasing pressure to diverge from values of openness, accessibility, fair pricing.

22:51 So as Hydra continues to grow, you know, what are the things that we need to do to make sure that

22:56 we're always emphasizing those values, that we don't diverge as we get bigger, more funding,

23:04 you know, things like that from outside or even inside pressures

23:07 within the company. So those are the things that I'm curious to know as you take the organization

23:12 from 10 people to 100 to 1,000, theoretically. That's very interesting. Yeah. How do you create

23:20 that compass that the organization can use for making its decision.

23:25 That compass and to continue to reinvest in that culture that emphasizes those tenants.

23:33 Absolutely.

23:34 I'd like to hear more about it.

23:36 Yeah.

23:38 Maybe we can make a question for more people in this interview series.

23:42 But yeah, you're absolutely right.

23:44 And something to write about for people that get this interview series. But yeah, you're absolutely right. And something to write about

23:45 for people that get their advice here.

23:49 Any sort of like additional remarks,

23:52 like for a closer that you might like to add

23:54 before it goes with like pointing people to the project

23:58 and to get started.

24:00 Closing remarks.

24:02 Well, first I just want to thank you for having me on. If people have

24:08 additional questions, yeah, just jump into our Discord. We're there. We have a lot of community

24:13 folks that I'm sure would be happy to chat, answer your questions. You know, data, we want everyone

24:21 to love data just as much as we do. And I think there's a real opportunity to bring great analytics to Postgres

24:29 in a way that, you know, they really haven't been before. So, you know,

24:34 Hydra is a data warehouse that's meant for everyone and it's accessible to,

24:38 to all people, wherever you're living. So like I mentioned,

24:41 it's free to start and I hope you'll give it a shot.

24:46 Phenomenal. Phenomenal. Thank you so much, Joe. Really appreciate this opportunity.

24:50 Oh, my pleasure.

24:52 Thanks.

24:53 All right. Cheers.

Library