Module Organization Guidelines
Should you organize modules vertically or horizontally? This week we take a look at another article by Gabriel Gonzalez, this time about organizing projects and packages.
Episode 46 was published on 2021-05-24.
>> Hello and welcome to the Haskell Weekly podcast. This is a show about Haskell, a purely functional programming language. I'm your host Taylor Fausak. I'm the director of software engineering at ACI Learning. And with me today is Cameron Gera. One of the engineers on my team. Thanks for joining me today, cam. Thanks
>> for having me today, Taylor, you know, it's uh, a beautiful sunny Friday at, uh, In the Gainesville, Florida area, which is where we are located. And I think a good day to day, we've got a phone article. Um, but you know, I think we also have some drama in the Haskell sphere, um, to something that I probably, or, you know, I don't even think you Taylor use this platform very often. Uh, but Freenode with, you know, an IRC channel more or less is going over some, some changes and creating some drama.
>> They are. Yeah, like you said, you know, neither of us are really IRC users, but the Haskell channel on Freenode is really popular. And there was some drama over the past week where the people who own free node kinda like changed hands or did something underhanded. And now everyone is leaving free node and moving over to this other place called, I want to say Libera or Libera. I'm not sure how to pronounce it. Um, so if you are an IRC user and you're on free node and you're. Where is the Haskell channel somewhere else now. So, uh, some other people on the internet can probably do a better job of explaining it than us. We just know that it happened.
>> We wanted to open it up for a little bit more banter
>> you know, but yeah, it's like I have slack. I have discord I'm on Reddit. I'm on Twitter. We use teams at work. Like I don't need another chat app. You don't
>> want Keybase and matrix and Libra or Libera? However
>> you say that, Nope. And Zulip and there's, there's too many.
>> There is. Well, you know, and we also, you know, on the podcast we've interviewed someone who also has another social media platform, uh, chat wise. Wisely.
>> If you haven't heard that episode, check it out.
>> Nice call back.
>> Yeah. You know, I always like to reference back to the
>> things we do. Yeah. The Haskell weekly cinematic universe. Yeah. I mean,
>> and speaking of we're actually taking a flashback from last week to some degree because we have another post by Gabe Gonzalez who's, we're going to talk about today and. Yeah. And I think Haskell weekly has a love affair with him, which is okay. Now I think he does put out regularly good post, which is nice to have for a site like Haskell weekly. So we can inform everyone and all of our listeners, uh, some, some cool stuff going on. So he kind of, uh, does three big ideas in this. Article. And this article is called module organization guidelines for Haskell projects. And like I said, there's three kind of big ideas that he kind of unwraps here. So we're going to just kind of jump right in. Uh, and this is something I think the first topic is one that honestly internally ITProTV and ACI Learning were divided on it as far as our code base is concerned because you know, when you have a web application with an API. You know, you tend to make a horizontal or our organization. Um, but here he talks about organizing modules vertically, not horizontally. Uh, so for us, if we were going to be creating libraries constantly and third-party SDKs more or less for other clients and yeah. Vertically would make sense. Um, For an API that has the same underlying types. Sometimes it's hard to do that.
>> Yeah. And it may be kind of hard to conceptualize what is meant by vertically and horizontally. I like to visualize it or think about it in terms of like vertical integration for a company where it's like, oh, we manage, you know, from farm to table or from widget to device or whatever it is. Uh, so it's like all of this stuff that you need to get. One thing done is all together in one slice. That is to me, Versus horizontal is you're going to split things up based on what they are rather than what they do. So it's like, okay, we're going to have all the types over here and we're going to have all the type classes over here. Um, and I feel like. For me, I tend toward the horizontal thing where I'm like, oh, well this is the type. I'll put it over here with all the other types. Uh, but at a certain size that kind of falls apart. And as you touched on in our code base, we have both. So we're diagonal.
>> Yeah. Well, I, I think your, uh, your analogy, there or explanation of it, uh, reminded me of something related more to farms, which is like a silo. You know, there's a silo like vertically that holds. Something, I don't really actually know the grain grain up, you know? And then there's like the troughs that the horses eat out of. So, no, I prefer to be a horse than a piece of grain. Cause I'm, can't eat gluten, so I don't want to always be irritated. So I'm going to go with the trough
>> style. Yeah. There you go. Trough versus silo. Yep. So
>> if you're a farmer,
>> let us know, let us know how wrong we got it. Um, and the example he gives is, uh, you know, dealing with like parsing and evaluating and type checking. So that makes sense because he's working on a language doll. So he naturally has all of these things that he's working on. But like you mentioned for our application, um, we're integrating with a bunch of third parties and we're also developing our own in-house stuff. So we end up a little bit of both where it's like, oh, we're talking to Intercom for our customer engagement platform. So all of the Intercom stuff goes over. But we have all of our like HTTP handlers in one namespace. So, so the handler part is horizontal, but the Intercom part is vertical. And I think it's, you know, in general, I agree with him. You should probably prefer verticality, but, um, you can make good use of both. Yeah. And I feel like
>> we've even kind of evolved through time with this, um, because yes, we were kind of vertically to some degree with our initial. HTTP server that we created, um, using Happstack we kind of kept the types and everything kind of siloed within, um, the, the structure. Then we've realized, okay, well now we have multiple HTTP servers that need to share these types. And, you know, we don't want to repeat them everywhere. Uh, but. We still have, like you said, like an Intercom integration and it makes more sense to push everything over. Um, and I'm actually dealing with that right now with, uh, practice labs. Cause I want to extract a lot of this stuff into its own kind of vertical rather than it being in the trough with everything else. Um, cause it's, it's hard to parse when it's in the middle of, you know, an, uh, database model versus, you know, Third-party response. Right? So it would be nice to clean that up. But I was looking at his kind of like module structure that he was talking about. And I was like, wow, we're probably more horizontal than we are vertical because we have a types module, we have not necessarily a lib module, but a that's where our, our source file is a lib file, more
>> or less. Um, and, uh, with regards to the types module, we do have one of those. We're moving away from it re exporting all of the types that are underneath it, which is one of the points he gives against this horizontal layout is that, uh, if you have like, you know, Types.* And then in other places that aren't types, you're going to want to import all of that altogether because it's convenient. But when, what ends up happening is if you change one type, everything has to get rebuilt because everything depends on all of the. And what we've moved toward instead is having more granular imports. So we'll put like each type in its own module and then import only the types we need. So that can get a little tedious when you're writing it. But then when you have to rebuild only the stuff that actually is affected by the changes you made, gets rebuilt, which is nice. We're
>> experiencing that tension of longer build times because you know, well, not necessarily build times but longer rebuild times because. At the bottom would get changed and then everything, you know, the rest of the tree would have to recompile and that was just painful, painful, and just frustrating. You're like, ah, I need now why, so thankfully we're, we're growing like many engineers do. Um, you know, so that's good
>> and engineering departments. It's like, we always say, you know, if you, if you look back at the code you wrote a year ago and you still think it looks. You probably haven't grown much in the meantime,
>> what, like an apple tree that hasn't produced any fruit. Yeah.
>> Um, but I also wanted to touch on one other thing. He mentioned here as motivation for pervert preferring the vertical orientation. And it's something you touched on as well, cam of, if there are pieces of your application or your, or whatever you're working on, that could be pulled out as separate packages. Those things are probably a vertical slice where it's like, it does everything it needs to. And, you know, it's like, okay, it's going to talk to Intercom or it's going to talk to whatever third-party or whatever, you know, library or driver. Um, and that is a compelling way to architect your application because then you can become sort of like a, you're just the glue between all of these little packages that you've built. Which is a nice way to maintain stuff,
>> right? Yeah. And it creates that nice interface. I think you and I were actually talking about that off air before we started just how much nicer, like, if you have just a good interface from a package, like experience, you can really work that into your application in a clear and readable way. And that doesn't muck it up with third-party information that doesn't really matter. The business logic that you have in place.
>> Exactly. That vertical slice forces you to think about. What your, what interface are you exposing versus with the types, the horizontal slice? You just say like, well, here's a bag of types. Good luck.
>> Good luck. Yeah. Yeah. I mean, in our like vert or when we were doing the. You know, overall big types re-export thing, you know, that file that holds all the re-exports is still just as, like super long and hard to really find, unless you're just doing a project wide search, like going through that file is like, oh wow, there's a lot here. So it's generally a little easier to like at least now we can just do fuzzy, find all this stuff without getting that file in the way.
>> Yeah, it's interesting that the vertical slice also encourages you to use better tools. And, in like, for us, you know, we would love to use HLS, but that's even, um, you know, like we, we don't need to tab nine gets the same thing. Fuzzy fine does the same thing. So like these general purpose tools that are built for anything can be leveraged to make writing vertically sliced Haskell code a lot nicer. I think that's everything I had to say about vertical versus horizontal. Um, you got anything else you want to move on to the next one?
>> I say let's, let's go on to
>> naming conventions. Yeah. So naming conventions are up next and, uh, this maybe is less relevant, I think to most people, because I feel like most Haskell developers probably aren't publishing packages. Like we've we published a handful. Um, and I personally have published a few, but I feel like most people just use existing packages. Hopefully, that's not too, too crazy of an idea.
>> Every Haskell developer doesn't create their own packages.
>> I would love it if everyone did, but if you're listening to this, please, you know, publish a package to Hackage, feel out what that whole process looks like. Um, it's fun and rewarding and you'll get a little more appreciation for what it takes to put together a good package. Right? Well,
>> and it seemed, I would completely agree with, uh, Gabes kind of position here on. When you're you have a package, you know, name, you know, keep the package name as close to the like module, the module name as possible. That way it's easier to understand. Oh, I'm importing, you know, foo-bar-baz, and I can access it at FooBarBaz.
>> Um, yeah. So the one exception to this, or not the one exception to like the strange difference here. Hackage package names are typically. Lowercased and connected with hyphens, which I often call kabob case. Cause it's like, you know, a kebab, um, and then Haskell package names are capitalized in camel case and separated with dots. So you can't have dots in package names. So it would make sense to replace those with hyphens. But, um, I mentioned this cause like a package, like, um, quick check, you know, the actual package name has a capital Q and a capital. Which is a little strange as far as packages go, but it is closer to the module name. Although in this case, the module name is test dot quick check. So those don't even match. So it's kind of the worst of both worlds, but so this is why Gabe wrote the article. Exactly. Um, I could, I could see the rationale for having a package named quick check with capital Q and C. And it exposing a module called quick check with capital Q and C cause then they would match. But for whatever reason, the community has decided no package names are lowercase and hyphenated, even though module names are camel case.
>> Good. Old kebab case. Yeah. I mean, I've become, you know, as probably a newer person to Haskell then you like, I've come to understand the reasoning for that. Not necessarily understand it, but accept it like, oh, okay. Um, 90% of the time, the package name is going to be a hyphenated version of the main module generally seems to be the practice. And I'm glad that Gabe kind of brought this up for the community to kind of just see and remind themselves, Hey, it's probably a better idea to keep this clear, because first of all, you have a lot, you know, if you're not the only user of this package, you're going to have people who are like confused or frustrated because it's just slightly different than what it should be. Um, or, you know, it is a really long module name, which we have really long module names because we have fairly nest nest data.
>> Um, yeah. There's no reason to make the primary interface for your package, a really long module name. And the example he gives is a perfect one for the pretty printer package used to the entry point, used to be data dot, text dot pretty print dot. And like, really you want to make everybody that uses your package import that mouthful. Why not just import pretty printer, which is what they switched to
>> that one's not camel cased.
>> So yeah, that's a weird,
>> uh, yeah. And then, you know, it seems to also help with naming clashes between packages,
>> right? Because if or the package name has to be unique, there's no way you can get around that. So if. Name your module after your package? It's probably going to be unique because somebody would have to be a jerk to right. You know, like if I wrote a package called, uh, you know, Taylor's pretty printer and I exposed a top level module called pretty printer, that would just be a jerk move on my part. So, you know,
>> gosh, such
>> a jerk Taylor. Yeah.
>> Awesome. Well, I think that's about, I have all I have to say on naming conventions. Is there anything else you have.
>> That's it for me too. Um, I, I agree with Gabe as per usual. Maybe that's why we talk about his posts so much on here. I just agree with everything he says, okay. If
>> you're listening, you know, reach out, we can get you on the
>> podcast. Yeah. We'd love to sit down for an interview.
>> Yeah. Maybe next, the next time we record a podcast since we're already on two in a row, just, just, trifecta
>> with an interview with you at the end. Yeah. He's going to slowly take over the whole podcast. It'll be the Gabe show.
>> And then we'll just be sending out the Gabe weekly newsletter.
>> Yep. You got it.
>> Nice. Um, all right, so what's, what's the next thing we're going to talk
>> about Taylor. So the last one is I think maybe the weirdest one. And when I hadn't thought about before he calls it, the God library stanza and he prefaces this with a caveat that it's only for. Proprietary projects. So stuff that you're not intending to publish on the Hackage or wherever. Um, but the idea is that instead of splitting up your package into the normal parts where you have a library and an executable and a test suite and benchmarks, instead you throw everything into the library. And then for all those other bits, all that you do is point to a particular part. So like for your executable, all you do is say, Hey, import, you know, my library dot executable and do that. And same thing for test suite import my library dot test suite, and do that. Um, we actually do this for our executable and I encourage everyone to do that of like, if you're going to write some type of CLI that you want to upload to. Put all of the actual business logic in a library. And that way, if people want to use it in other Haskell programs, they can. And the perfect example of this is HLint. They expose, or they Neal exposes the entire, um, API as, you know, just a regular module that you can use. So if you want to use it on the command line, you can do that. If you want to use it in your library, you can do that too. Um, but what Gabe is suggesting here is even more extreme, right? He wants you to put your entire test suite into your library and that's crazy, right, Kim with like, why would you do that?
>> Well, I'm in why not at this point. I mean, he, he, he says, you know, don't do it for open source projects, but if you have a proprietary project you're not putting out there, like, it seems to be a better move. And I'm sorry, I'm just trying to,
>> catch up a little bit more on it. You're good. So, so I think the main benefit he poses here is that for the cabal command line tool, um, if you want to bring up a rebel for your project, cabal, can't load your library and your executable or your test suite or whatever else at the same time, it can only pick one component to do at. So, if you want to like, run your test suite in the REPL, because you're editing it and you want to get some quick feedback, you can do that. But as soon as you make a change to your library and you go and reload your test suite, it's not going to pick that change up. You have to close the whole REPL and start it up again, which is a huge pain in the butt. Right.
>> Right. So, yeah, which I mean that, that would keep the rebel
>> development. Yeah. And, and I'm saying rebel, but this applies to like GHC ID as well, because that's just a rebel under the hood, under the hood. Yeah. Um, and I mentioned that for us, we have our executable implemented like this already, but we don't do this for our test suite. And I think one of the reasons why is that we use stack as our build tool and stack actually. Load up the REPL with your library and your test suite at the same time. So we haven't run into this particular limitation,
>> but if you're using Cabal would be worth a shot.
>> Yeah. And it'll also give you quicker recompiles on your test suite. So normally if you changed anything in your library at all, your test suite naturally depends on your library. So the whole thing has to get rebuilt because that entire component has changed. So it assumes everything is busted. If you have, you know, file a and file a, has a test associated with it. And same for B. Then if you change a B and BS test, won't need to be rebuilt. So that feedback loop will be quicker. Nice.
>> Yeah. It seems, seems like a useful thing.
>> Yeah. Yeah. I, I am excited to try it out. I've never set up any of my projects. With the test suite, like mixed in with the regular code. So I want to try that out and see how it goes. Yeah. Maybe we can
>> throw it into our, since our test suite always seems to get, be getting re compiled as we add more tests, it's more tedious,
>> you know? Yeah. Tedious, driven development, TDM driven development.
>> Perfect. That's the new TDD.
>> Um, but yeah, and, uh, like our test suite frequently. It doesn't like it's got a hundred or 200 modules in the test suite, and usually you haven't touched any source files that would cause the test suite to get rebuilt, but the whole thing gets rebuilt anyway. So that's a bit annoying. It'd be nice to avoid that.
>> Yeah. It could be a quality of life improvement for us here at ACL.
>> Sure. All right. So we've reached the end here of yet another wonderful Gabe Gonzales article.
>> You know, I was just going to ask you as my shirt, twin, you know, with which of these three was your, what's your favorite?
>> Ooh, I think my favorite is the vertical versus horizontal. And, uh, I just feel like it, it's such a good way to think about architecting your application, even though we don't, you know, like rigidly adhere to it all the time. I think that any given vertical slices, probably going to be more useful than the, the corresponding horizontal slice. So like, if you just give me here's all the Intercom stuff to keep using that one, as an example, I'll be able to look at that and figure out what's going on. Versus if you say here's all of our types. Okay.
>> Oh, thanks. Yeah. It gives you a little bit more clarity of what should be happening here. The Intercom slice deals and talks to Intercom
>> to return. Which one's your fave cam? Uh, I
>> would also say vertical versus horizontal. I think it's something that we've kind of wrestled with in the past. Just of like figuring out sometimes we, we kind of have pushes towards right let's silo stuff, then there's times where they're like, okay, yeah, let's put it in a more general spot that more things can use. And I, I just think for us, it depends on what that is. Is it a new API integration with a third party? Is it a new way of importing and exporting types? Is it, you know, a new way of querying the database? Like what's, you know, so I think for us, it just depends on, I mean, sometimes it's almost like they're their own vertical, like, oh, this type is a vertical, but it's not because it's used across multiple, you know, it's like a silo with a bunch of tubes going to other places.
>> I don't think that's how we're really stretching the metaphor here.
>> You know, I'm doing what I can here, you know, but no, I think, uh, I think that would be mine as well.
>> Cool. Yeah. And, and again, just to really underscore this, like, um, by designing those vertical slices, we, in the course of development, we have asked ourselves, what would this look like if we turned it into a separate, like, And we don't always follow through on that. You know, we don't always end up publishing another library as a result of that, but it's still useful to go through those steps. And sometimes we do end up publishing a library and then, you know, it's actually useful. It's not just a bag of types or, or whatever else. It's like, oh, you can use this to talk to. We haven't published in Intercom library, but you can use this to talk to, um, Sentry or Recurly or something like that. So you're saying
>> WWLD what would a library.
>> Oh, yeah, there you go. I was thinking WWGGD what would Gabe Gonzalez do
>> that's also a good one. Uh, so look out for on the Haskell weekly website for t-shirts. WWGGD and WWLD. So that should be coming to a Haskell weekly near you
>> soon. All right. Well, that's all I've got, you got anything else, cam? Nah, I think that's about it. Cool. Well, that will do it for us this week. Uh, thank you so much for listening to the Haskell weekly podcast. I have been your host Taylor Fausak and with me today was Cameron Gera. If you want to find out more about us, everything you want to know is at our website and our website is HaskellWeekly.News. I just realized
>> that I didn't say my name once, so I don't have to introduce myself, but this is nice. But anyways, Haskell Weekly is brought to you, uh, by it pro TV and ACI Learning. Also our employer, they would like to offer you 30% off the lifetime of your subscription at ITPro.TV by using the promo code HaskellWeekly30 at checkout. And if you're not interested in paying for a membership, we also offer a free one that will get you access to some great content. But I think that doesn't about blah, blah. Blah-blah-blah. Wow. I can't talk today, but that's okay. Uh, I think that about does it for us. Thank you for joining us on the Haskell with the podcast and we'll see you next week.