Telegram Bot In Scala 3 With Bot4s + http4s + Doobie for CI notifications

In this article, I want to show you how to brew a simple Telegram bot using Scala 3, cats, library and also discuss the current state of the Scala ecosystem regarding Scala 3 support. bot4s Goal Let’s say you have a team that uses telegram for communications, you also use some CI system e.g. TravisCI, and now you want to have a channel to receive notifications about build statuses. You did a little research and you have found articles like explaining how to add a simple script that can be triggered after the build to send notifications to the specified telegram chat. this Well that’s a solution, but you are not satisfied with it, because you don’t want to replicate that script for each of your microservice repositories, you want to define a single service that will handle notifications, a single place to change your notification format and a single place to store telegram credentials, there is also a possibility of switching to another CI system soon, so you really want to protect yourself from refactoring dozens of ad-hoc scripts in the future. To solve all of those problems you can create your own simple telegram bot, and implementing it with Scala 3 is also a hell of fun! Prerequisites We will use Scala 3 and SBT (Scala Build Tool) for that purpose. You should have SBT installed on your computer, see this for more details. Choose your favorite editor or IDE and you are good to go! (I’m using VSCode + Metals) link Requirements Before jumping straight into coding let’s discuss what we want in particular from our little bot. Ok. We know there is no native support of telegram notifications in our CI system, but it has configurable webhook support, so we need our bot to expose some webhook endpoint so CI can call it after the build. After the webhook is triggered we need to transform the incoming message into a telegram message and send it somehow to the appropriate chat. And we also want our bot to be able to notify different chats about different project builds, so we need to create and persist a mapping from the repo identity to chat. Let’s gather all requirements for our bot into a list: Webhook endpoint for CI notifications Webhook request transformation into an appropriate, human-readable Telegram message Persisting mapping from repository Id to chat Id. Sending notifications upon receiving to chat accordingly to persisted mapping. Make it extensible enough to be able to add support for different CI systems. Code! For the sake of simplicity I obfuscated some parts of code used in the application like imports, factory methods, main function, etc. Please see the full code in the . Github repository API and Models Before implementing the bot itself let’s start by defining our model and interfaces. Assuming we will have a class that will implement a bot functionality and speak to telegram’s bot API we want to abstract over notifying functionality so higher-level code would not interact with bot implementation directly but throughout the abstraction. For this purpose, we will define a trait: BuildInfoNotifyer trait BuildInfoNotifyer[F[_], BI]: def notify(buildInfo: BI): F[Unit] Pay attention to , this signals we are using the tagless-final encoding. In a nutshell, it helps to abstract over the effect type, e.g. IO, Future, Task, etc, if you want to know more about this pattern please see . We also use the new python-ish braceless syntax of Scala 3. F[_] this article Now, we need a model class to pass information through our application. final case class BuildResultNotification( repoId: RepoId, subscriberId: SubscriberId, buildNumber: BuildNumber, commitMessage: CommitMessage, authorName: AuthorName, branch: BranchName, resultStatus: ResultStatus, failed: Boolean, buildUrl: BuildUrl ) extends HasSubscriberId This case class you see above would be our general notification data carrier for all CI integrations, it will be passed to bot instance to send messages. Field types of models in this project are implemented as a simple type aliases for the sake of simplicity. If you want to be an FP PRO you can use a combination of and types to disambiguate and validate your types. newtypes refined Additionally, it would be nice to define a way to convert into a Telegram message and to separate this concern into an appropriate abstraction. The best way to do this is a type-class, let’s see how we can create one in Scala 3: BuildResultNotification trait TelegramMarkup[M]: extension (m: M) def markup: String extension (m: M) def parseMode: ParseMode.ParseMode More on that syntax in . official doc Now we can put type-class instance into companion object: TelegramMarkup BuildResultNotification object BuildResultNotification: import unindent._ given TelegramMarkup[BuildResultNotification] with extension(brn: BuildResultNotification) def markup: String = val result = if (brn.failed) "Failed" else "Succeeded" i""" *Build* [#${brn.buildNumber}](${brn.buildUrl}) *$result* *Commit*: `${brn.commitMessage}` *Author*: ${brn.authorName} *Branch*: ${brn.branch} """ extension(brn: BuildResultNotification) def parseMode: ParseMode.ParseMode = ParseMode.Markdown We have put this instance into a companion object so it would be accessible by scala compiler implicit search when needed. Now let’s get to the web model classes. final case class TravisCIBuildResultInfo( id: BuildId, number: BuildNumer, status: Status, result: Result, message: Message, authorName: AuthorName, branch: BranchName, statusMessage: StatusMessage, buildUrl: BuildUrl, repository: Repository ) final case class Repository(id: RepositoryId) object TravisCIBuildResultInfo: val TravisCiFailedStatuses = Seq("failed", "errored", "broken", "still failing") extension (bri: TravisCIBuildResultInfo) def asNotification(subscriberId: SubscriberId): BuildResultNotification = BuildResultNotification( repoId = bri.repository.id.toString, subscriberId = subscriberId, buildNumber = bri.number, commitMessage = bri.message, authorName = bri.authorName, branch = bri.branch, bri.statusMessage, failed = TravisCiFailedStatuses.contains(bri.statusMessage.trim), bri.buildUrl ) class serves as a TravisCI webhook request body representation that we can convert to using Scala 3 feature called in a nutshell, this allows us to add methods to a class without changing it, see more info in the . TravisCIBuildResultInfo BuildResultNotification extension methods, official documentation Finally, we need to model our repository-to-chat connection, for that we will introduce a new entity - : SubscriberInfo final case class SubscriberInfo(subscriberId: UUID, chatId: Long) We will see how this model will be used in our application later on. CI Webhook Let’s start our concrete implementation by fulfilling our first requirement - the webhook endpoint. We will implement a simple route using DSL: http4s class BotWebhookController[F[_]: Concurrent] private ( notifier: BuildInfoNotifyer[F, BuildResultNotification], logger: Logger[F] ) extends Http4sDsl[F]: def ciWebHookRoute: HttpRoutes[F] = import org.http4s.circe.CirceEntityDecoder._ HttpRoutes.of[F] { case req @ POST -> Root / "travis" / "build" / "notify" / "subscriber" / UUIDVar(subscriberId) => for form Concurrent[F] .fromEither(decode[TravisCIBuildResultInfo](payload)) .flatMap(webhookData => notifier.notify(webhookData.asNotification(subscriberId))) } result <- Ok("ok") yield result } method will return a route that contains a POST HTTP method that receives a webhook notification from TravisCI. You don’t need to dive deep into the details of this function, but to sum things up, this method will get JSON payload out of the request body, parse it into an instance of case class, and then after transforming it into a general notification format webhook will call with this data. ciWebHookRoute BuildResultInfo BuildInfoNotifyer#notify Persistence with Doobie As the last step before we start working on the bot let’s define a persistence layer that will help us store info about notification subscribers. PostgreSQL was chosen as a database because it’s fairly easy to deploy and manage on platforms like Heroku and it is also sufficient for our task where we need to query subscribers both by subscriberId and by chatId. First, we will need to have an interface that will represent our data storage: trait SubsriberInfoRespository[F[_]] { def save(si: SubscriberInfo): F[Unit] def find(subscriberId: SubscriberId): F[Option[SubscriberInfo]] def find(chatId: ChatId): F[Option[SubscriberInfo]] def delete(chatId: ChatId): F[Unit] With we declared a bunch of methods for saving, deleting and searching subscriptions. Good, now we need to implement it. SubsriberInfoRespository Concrete implementation of this trait in will look like this: doobie class PostgreSubsriberInfoRespository[F[_]: MonadCancelThrow](xa: Transactor[F]) extends SubsriberInfoRespository[F]: override def save(si: SubscriberInfo): F[Unit] = sql"""INSERT INTO subscriber("chatid", "subscriberid") VALUES(${si.chatId}, ${si.subscriberId})""".update.run .transact(xa) .void override def find(subscriberId: SubscriberId): F[Option[SubscriberInfo]] = sql"SELECT subscriberid, chatid FROM subscriber WHERE subscriberid = $subscriberId" .query[SubscriberInfo] .option .transact(xa) override def find(chatId: ChatId): F[Option[SubscriberInfo]] = sql"SELECT subscriberid, chatid FROM subscriber WHERE chatid = $chatId" .query[SubscriberInfo] .option .transact(xa) override def delete(chatId: ChatId): F[Unit] = sql"DELETE FROM subscriber WHERE chatid = $chatId".update.run .transact(xa) .void Doobie has a simple API that covers all of our scenarios perfectly, it uses free monad that can be thought of as program description or “how” you want to query your DB, it’s not going to execute anything. Then, is an entity that knows how to manage DB connections, and using this entity we can transform to our Monadic type . ConnectionIO Transactor ConnectionIO F Telegram Bot Now everything is set in place for the bot implementation. The class that implements the bot is too big to fit one code block so I’m gonna put it in a series of blocks and comment on each separately. We will start from a class declaration: class CITelegramBot[F[_]: Sync, N <: HasSubscriberId: TelegramMarkup] private ( botAppConfig: BotAppConfig, subscriberInfoRepository: SubsriberInfoRespository[F], backend: SttpBackend[F, Any], logger: Logger[F] )(using MCT: MonadCancelThrow[F]) extends TelegramBot[F](botAppConfig.telegramApiConfig.token, backend), Commands[F], Polling[F], Callbacks[F], BuildInfoNotifyer[F, N]: First of all, our class need to extend class called . This class requires a telegram token and another abstraction called , which is an HTTP client that will be used to call telegram API (for more info about STTP see ). is a type-class that allows us to raise errors with our F monad. Traits like , and that are implemented by our are all parts of the bot4s API and they allow us to add commands, callbacks and poll telegram API. bot4s TelegramBot SttpBackend official doc MonadCancelThrow Commands Callbacks Polling CIBuddyBot Let’s add a couple of commands to give users a way to subscribe/unsubscribe from notifications: onCommand("subscribe") { implicit msg => for _ for _ <- subscriberInfoRepository.delete(msg.chat.id) _ (for msg OptionT(subscriberInfoRepository.find(msg.chat.id)) .foldF( logger.warn( s"No subscriptions found for chat: chatId - ${msg.chat.id}" ) ) { subscriberInfo => reply(replyText.format(subscriberInfo.subscriberId.toString)).void } } ).getOrElse(logger.warn("Message or callback data was empty")) } This function will register a new callback handler that will react when bot user will press the button. In our case earlier in “subscribe“ command we have implemented functionality that will respond to the user with a keyboard with all available CI tools options and this callback handler will receive a tool option that the user has chosen. Upon callback receive, this handler will search for chosen CI build tool option in a list of available ones and then it will get the subscriptionId using the current chatId. onCallbackQuery It will respond to the user with a message with a format: Use this url in your travisci webhook notification config: 'https://bot-host-url.com/travis/build/notify/subscriber/$subscriberId' Users can then use provided webhook URL to configure their build. Alright, we have implemented a full subscription flow, now the last thing left to implement is actual notifications: override def notify(notification: N): F[Unit] = OptionT(subscriberInfoRepository.find(notification.subscriberId)) .foldF( logger.warn( s"Subscriber not found : subscriberId - ${notification.subscriberId}" ) ) { subscriber => request( SendMessage( subscriber.chatId, notification.markup, parseMode = Some(notification.parseMode) ) ).void } is an implementation of trait, this function will send a message to a Telegram chat according to . Notice how we use the information from notification. We don’t extract it directly, we use a function that comes from notify BuildInfoNotifyer subscriberId markup type-class that we declared earlier and we also said that our abstract type has an instance of it using . TelegramMarkup N context bounds Good, now as bot implementation is done, we can start testing it (bot run/deployment details are described in the projects’ file), check out the next section to see the results! README Results Now let’s see the actual results in Telegram! We will assume we already configured our bot (see for more instructions) and deployed the bot application somewhere. this doc First, we need to subscribe to notifications: And now assuming we configured our CI to send webhook notifications let’s see an example of such notification received in a chat: Perfect! Our CI bot works as expected, we were able to subscribe and receive a message from a webhook! The only thing that is not shown here is how to configure a webhook on the CI side, but that depends on the specific CI build configuration (for TravisCI see this ). doc Notes and comments on Scala 3 support At the time of writing this article (October 2021) I have encountered a by ecosystem: bunch of issues regarding Scala 3 support bot4s doesn’t support Scala 3 yet and it also has a dependency on cats which means that I can’t use Scala 3 versions of other libraries dependent on cats. So I was forced to mark all main dependencies as .cross(CrossVersion.for3Use2_13) Consequently, I was not able to use macro-based functions from libraries like doobie, log4cats, circe because in my case they were all built for Scala 2.13 and you can’t use 2.13 macros in Scala 3 project. So for example instead of writing I was forced to use functions without a macro like in the case of doobie I’ve backported Scala3 macros inside of the project because without them it’s just unusable. In the case of circe I defined decoder by hands instead of using auto derivation which is based on macros. Slf4jLogger.getLogger[F] Slf4jLogger.fromName[F](“CiBot“) IntelliJ IDEA doesn’t support Scala 3 so well yet, I’ve started working on this project in IDEA, but encountered some compilation issues I was not able to overcome, so I resorted to VSCode + Metals and was surprised how good it is nowadays, so I would definitely recommend this combination for your Scala 3 project! These are all the major problems I’ve encountered, mainly lack of Scala 3 support in some libraries, but those are mitigate-able. If you encounter such issues remember that you always can use Scala 2.13 version of libraries in your Scala 3 project which is great. If that library has macros search for alternative macros-less functions inside of that library or backport/port macros by hands to Scala 3 macros. : On the bright side You can use 2.13 libraries in Scala 3, that is exactly what enabled me to produce this application! This means you could already migrate your application because all major libraries compile to 2.13 (yes, even Spark!), although I would recommend not to migrate at least until Scala 3.2.0 release because there are some that this version will solve forward compatibility problems. talks Recently Scala 3.1.0 was released and I bumped my Scala version in the file and it just works without any changes in code!! From now on Scala minor versions are backward compatible which is great although this version is not forward compatible, which means you can’t use a 3.1.0 library in a 3.0.2 project, you will be forced to bump your Scala version, but as already mentioned there is a chance that Scala team will try to make version 3.2.0 fix this issue. build.sbt Overall I think Scala made good progress in terms of language usability and compatibility between language versions which makes it more comfortable to use. It needs just a bit more time to bump the ecosystem and make it even more compatible, but in the end, it will be a great choice for your next application, much better than previous versions of Scala. * Bot Deployment Notes * I’ve deployed my instance of bot using platform, it is fairly simple and straightforward. heroku Bot project is packaged using so you could use that for your deployments, see more info in the bot project README file. sbt-native-packager Summary In this article, we have seen how to implement a working application in Scala 3! We satisfied all our initial requirements and built our application in an extensible and persistent way. We also have shown how to build a convenient solution for your Telegram notifications. Not only it was fun to code in Scala 3 and it was also pretty efficient, with a new syntax Scala developer day-to-day work is greatly simplified. We also discussed how well Scala 3 is compatible with previous and future versions and in my opinion, great work is done in that field and I really appreciate Scala team effort they have put into this. Useful Links Bot code repository itself - Example of a shopping cart application built with modern Scala FP stack, also includes a great number of best practices examples. Tagless-final based application example - great book by Gabriel Volpe, describes best practices of building modern FP applications in Scala Practical FP In Scala - I’ve used this library to deal with multiline strings unindent TravisCI webhook config doc