In systems that need to handle large volumes of background data — such as billing routines — it’s common to fall into the trap of writing temporary scripts or running manual processes. That’s exactly what we set out to avoid. In this article, I share how Oban helped us structure a resilient and scalable job system, becoming a core part of our billing process at Nextcode. Oban The challenge: complex and recurring billing routines Our scenario involved: Processing thousands of usage logs daily
Fetching logs from different apps and databases
Applying client-specific and service-specific rules
Aggregating and generating logs
Ensuring safe retries in case of failures
Horizontal scaling without losing traceability
Storing data in optimized databases for querying
Running these routines manually or with one-off scripts was risky, especially with no logging or fallback in case of failure Processing thousands of usage logs daily Fetching logs from different apps and databases Applying client-specific and service-specific rules Aggregating and generating logs Ensuring safe retries in case of failures Horizontal scaling without losing traceability Storing data in optimized databases for querying Running these routines manually or with one-off scripts was risky, especially with no logging or fallback in case of failure That’s when Oban came into play. Oban Why we chose Oban Besides being built in pure Elixir, Oban gave us everything we needed: ✅ PostgreSQL persistence ✅ Automatic retries with backoff ✅ Job deduplication with uniqueness control ✅ Dashboard support via oban_web (yet to try it) oban_web ✅ Distributed execution with isolated queues and concurrency ✅ Native Ecto integration and flexibility Installation Getting started is simple. Just add the dependency to your mix.exs: mix.exs def deps do
  [
    {:oban, "~> 2.17"},
  ]
end def deps do
  [
    {:oban, "~> 2.17"},
  ]
end Then, configure your repo and supervision tree: # config/config.exs
config :my_app, Oban,
  repo: MyApp.Repo,
  plugins: [
    {Oban.Plugins.Pruner, max_age: 86_400},
  ],
  queues: [
    mongodb_daily_log: 1
  ] # config/config.exs
config :my_app, Oban,
  repo: MyApp.Repo,
  plugins: [
    {Oban.Plugins.Pruner, max_age: 86_400},
  ],
  queues: [
    mongodb_daily_log: 1
  ] # application.ex
children = [
  {Oban, Application.fetch_env!(:my_app, Oban)}
] # application.ex
children = [
  {Oban, Application.fetch_env!(:my_app, Oban)}
] Our worker: MongodbDailyLog MongodbDailyLog Here’s how we structured one of our workers to process daily logs for billing: defmodule MyApp.Job.MongodbDailyLog do
  use Oban.Worker,
    queue: :mongodb_daily_log,
    max_attempts: 2,
    unique: [
      fields: [:args],
      states: [:available, :scheduled, :executing],
      period: 60
    ] defmodule MyApp.Job.MongodbDailyLog do
  use Oban.Worker,
    queue: :mongodb_daily_log,
    max_attempts: 2,
    unique: [
      fields: [:args],
      states: [:available, :scheduled, :executing],
      period: 60
    ] What does this do? Assigns a specific queue for the job
Limits each job to 2 attempts
Enforces uniqueness to avoid duplicate executions with the same arguments Assigns a specific queue for the job Limits each job to 2 attempts Enforces uniqueness to avoid duplicate executions with the same arguments Automatic scheduling and safe execution Our job can be triggered either automatically or on-demand. We use Timex to handle dates and define the processing window: Timex def perform(%Oban.Job{args: %{"date" => %{"day" => d, "month" => m, "year" => y}, "only_logs" => only_logs}}) do
  date = Timex.to_date({y, m, d})
  gte = Timex.to_datetime(date, "America/Sao_Paulo")
  lt = Timex.shift(gte, days: 1)

  job_impl().run(%{gte: gte, lt: lt}, only_logs)
end def perform(%Oban.Job{args: %{"date" => %{"day" => d, "month" => m, "year" => y}, "only_logs" => only_logs}}) do
  date = Timex.to_date({y, m, d})
  gte = Timex.to_datetime(date, "America/Sao_Paulo")
  lt = Timex.shift(gte, days: 1)

  job_impl().run(%{gte: gte, lt: lt}, only_logs)
end Default execution for previous day if no date is specified: def perform(%Oban.Job{args: %{}}) do
  %{day: d, month: m, year: y} = Timex.shift(Timex.today(), days: -1)
  %Oban.Job{args: %{"date" => %{"day" => d, "month" => m, "year" => y}, "only_logs" => false}}
  |> perform()
end def perform(%Oban.Job{args: %{}}) do
  %{day: d, month: m, year: y} = Timex.shift(Timex.today(), days: -1)
  %Oban.Job{args: %{"date" => %{"day" => d, "month" => m, "year" => y}, "only_logs" => false}}
  |> perform()
end Deduplication and execution We run the job with deduplication to prevent accidental duplicates: def run(%Date{} = date \ Timex.shift(Timex.today(), days: -1), only_logs \ false) do
  %{day: day, month: month, year: year} = date

  job =
    %{date: %{day: day, month: month, year: year}, only_logs: only_logs}
    |> NextID.Job.MongodbDailyLog.new()

  with {:ok, %Oban.Job{conflict?: true}} <- Oban.insert(job) do
    {:error, :job_already_exists}
  end
end def run(%Date{} = date \ Timex.shift(Timex.today(), days: -1), only_logs \ false) do
  %{day: day, month: month, year: year} = date

  job =
    %{date: %{day: day, month: month, year: year}, only_logs: only_logs}
    |> NextID.Job.MongodbDailyLog.new()

  with {:ok, %Oban.Job{conflict?: true}} <- Oban.insert(job) do
    {:error, :job_already_exists}
  end
end Historical processing? No problem. Need to backfill historical data? We created a method that recursively schedules jobs by date: def run_history(%Date{} = date \ Timex.today()) do
  case Timex.before?(date, ~D[2021-08-01]) do
    false ->
      run(date, true)
      run_history(Timex.shift(date, days: -1))
    true -> :ok
  end
end def run_history(%Date{} = date \ Timex.today()) do
  case Timex.before?(date, ~D[2021-08-01]) do
    false ->
      run(date, true)
      run_history(Timex.shift(date, days: -1))
    true -> :ok
  end
end Results We automated daily log processing with high reliability
Achieved horizontal scalability by splitting queues by task type
Eliminated duplication issues while maintaining traceability
Reduced manual work and reprocessing effort
Gained full visibility and control over all jobs — for now, straight from PostgreSQL We automated daily log processing with high reliability Achieved horizontal scalability by splitting queues by task type Eliminated duplication issues while maintaining traceability Reduced manual work and reprocessing effort Gained full visibility and control over all jobs — for now, straight from PostgreSQL Conclusion Oban turned out to be a robust, easy-to-implement, and well-integrated solution within the Elixir ecosystem. Today, it’s a foundational part of our billing system — and we’re already expanding its use to other parts of the product. If you’re dealing with critical background processing like billing, I highly recommend giving it a try. 📚 Official Oban repo: github.com/oban-bg/oban github.com/oban-bg/oban 💡 About Nextcode: https://nxcd.com.br 💬 Want to chat about how we’re using it? Reach out!

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

How We Manage Billing Workflows in Elixir Using Oban

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

17 Stories To Learn About Elixir

3 Things I Learned in 1 Year Working with Functional Programming

A free introduction to Elixir, OTP, Ecto, and Phoenix

A Hitchhiker’s Tour of the BEAM

A Look at Using Elixir Streams, Elasticsearch & AWS S3

17 Stories To Learn About Elixir

3 Things I Learned in 1 Year Working with Functional Programming

A free introduction to Elixir, OTP, Ecto, and Phoenix

A Hitchhiker’s Tour of the BEAM

A Look at Using Elixir Streams, Elasticsearch & AWS S3

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps