In many cases, developers must use transactions when performing various operations on the server. For example - a transfer of money, or other measurable value, and much more.
With such operations, I really do not want to receive an error that will interrupt the process and violate the data integrity.
What is a "transaction" anyway? Wikipedia says:
"A database transaction symbolizes a unit of work performed within a database management system (or similar system) against a database, and treated in a coherent and reliable way independent of other transactions. A transaction generally represents any change in a database."
Transactions in a database environment have two main purposes:
Now, consider a situation where an error can occur, leading to very unpleasant consequences if you do not use transactions.
I made a small project in which there are two entities:
Users can transfer money to each other. When transferring, the sufficiency of the amount on the balance of the sender is checked, as well as many other checks. If a situation occurs when the money has been debited from the sender's balance but not transferred to the recipient's account, or vice versa, we will see either a very sad, angry person, or we will not see a very happy one (depends on the transfer amount).
Great, with the fact that transactions are important and need to be sorted out (hopefully everyone agrees with this). But how do you apply them?
First, let's look at the options for queries with errors and without errors that will occur if you use PostgreSQL.
The usual set of queries without errors:
// ...
SELECT "User"."id" AS "User_id", "User"."name" AS "User_name", "User"."defaultPurseId" AS "User_defaultPurseId"
FROM "user" "User"
WHERE "User"."id" IN ($1)
START TRANSACTION
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
START TRANSACTION
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
By the way - I did not write this request by hand, but pulled it out of the ORM logs, but it reflects the essence.
Everything is pretty simple and straightforward. To build the queries, TypeORM was used, which we will return to a little later.
The ORM and Postgres settings are set by default, so each operation will be performed in its own transaction, but to take advantage of this advantage, you need to write one query in which all the logic associated with the database will take place at once.
Below is an example of the execution of multiple queries executed in one transaction:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
The key difference with the previous example of requests is that in this case all requests are executed in one transaction, and therefore, if an error occurs at some stage, the entire transaction will be rolled back with all the requests inside it.
More or less like this:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
ROLLBACK
And here, by the way, is the code that produced all the previous SQL queries. It contains a flag, when set, an error occurs at the most inopportune moment:
// ...
async makeRemittance(fromId: number, toId: number, sum: number, withError = false, transaction = true): Promise<RemittanceResultDto> {
const fromUser = await this.userRepository.findOne(fromId, { transaction });
const toUser = await this.userRepository.findOne(toId, { transaction });
if (fromUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(fromId));
}
if (toUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(toId));
}
if (fromUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(fromId));
}
if (toUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(toId));
}
const fromPurse = await this.purseRepository.findOne(fromUser.defaultPurseId, { transaction });
const toPurse = await this.purseRepository.findOne(toUser.defaultPurseId, { transaction });
const modalSum = Math.abs(sum);
if (fromPurse.balance < modalSum) {
throw new Error(NOT_ENOUGH_MONEY(fromId));
}
fromPurse.balance -= sum;
toPurse.balance += sum;
await this.purseRepository.save(fromPurse, { transaction });
if (withError) {
throw new Error('Unexpectable error was thrown while remittance');
}
await this.purseRepository.save(toPurse, { transaction });
const remittance = new RemittanceResultDto();
remittance.fromId = fromId;
remittance.toId = toId;
remittance.fromBalance = fromPurse.balance;
remittance.sum = sum;
return remittance;
}
// ...
Fine! We saved ourselves from losses or very upset users (at least in matters related to money transfers).
What's next? What other ways are there to write a transaction? It just so happened that the person whose article you are currently reading (this is me) really loves one wonderful framework when he has to write a backend.
The name of this framework is Nest.js. It works on the Node.js platform, and the code in it is written in Typescript. This great framework has support, almost out of the box, for the very TypeORM.
Which (or which?) I, as it happens, also really like. I didn't like only one thing - a rather confusing, as it seems to me, overly complicated approach to writing transactions.
This is the official example for writing transactions:
import { getConnection } from 'typeorm';
await getConnection().transaction(async transactionalEntityManager => {
await transactionalEntityManager.save(users);
await transactionalEntityManager.save(photos);
// ...
});
Second way to create transactions from the documentation:
@Transaction()
save(user: User, @TransactionManager() transactionManager: EntityManager) {
return transactionManager.save(User, user);
}
In general, the point of this approach is as follows: you need to get a
transactionEntityManager: EntityManager
- an entity that will allow you to execute queries within a transaction. And then use this entity for all actions with the base. Sounds good, as long as you don't have to deal with using this approach in practice.To begin with, I don't really like the idea of injecting dependencies directly into the methods of service classes, as well as the fact that the methods are written in this way become isolated in terms of using the dependencies injected into the service itself.
All the dependencies necessary for the method to work will have to be dropped into it. But the most annoying thing is that if your method calls other services embedded in yours, then you have to create the same special methods in those third-party services. And pass
transactionEntityManager
in them. At the same time, it should be borne in mind that if you decide to use the approach through decorators, then when you transfer the
transactionEntityManager
from one service to the second, and the method of the second service will also be decorated - in the second method you will receive the transactionEntityManager
that is not passed as a dependency, and the one that is created by the decorator, which means two different transactions, which means unfortunate users.Below is the code for a controller action that handles user requests:
// ...
@Post('remittance-with-typeorm-transaction')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTypeOrmTransaction(@Body() remittanceDto: RemittanceDto) {
return await this.connection.transaction(transactionManager => {
return this.appService.makeRemittanceWithTypeOrmV1(transactionManager, remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
});
}
// ...
In it, we need to have access to the
connection
object to create a transactionManager
. We could do as the TypeORM documentation advises - and just use the getConnection
function as shown above:import { getConnection } from 'typeorm';
// ...
@Post('remittance-with-typeorm-transaction')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTypeOrmTransaction(@Body() remittanceDto: RemittanceDto) {
return await getConnection().transaction(transactionManager => {
return this.appService.makeRemittanceWithTypeOrmV1(transactionManager, remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
});
}
// ...
But it seems to me that such code will be more difficult to test, and this is simply wrong (great argument). Therefore, we will have to pass the
connection
dependency into the controller constructor. It's very lucky that Nest allows you to do this by simply describing the field in the constructor with the appropriate type:@Controller()
@ApiTags('app')
export class AppController {
constructor(
private readonly appService: AppService,
private readonly connection: Connection, // <-- it is - what we need
) {
}
// ...
}
Thus, we come to the conclusion that in order to be able to use transactions in Nest when using TypeORM, it is necessary to pass the
connection
class into the controller / service constructor, for now we just remember this.Now let's look at the
makeRemittanceWithTypeOrmV1
method of our appService
:async makeRemittanceWithTypeOrmV1(transactionEntityManager: EntityManager, fromId: number, toId: number, sum: number, withError = false) {
const fromUser = await transactionEntityManager.findOne(User, fromId); // <-- we need to use only provided transactionEntityManager, for make all requests in transaction
const toUser = await transactionEntityManager.findOne(User, toId); // <-- and there
if (fromUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(fromId));
}
if (toUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(toId));
}
if (fromUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(fromId));
}
if (toUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(toId));
}
const fromPurse = await transactionEntityManager.findOne(Purse, fromUser.defaultPurseId); // <-- there
const toPurse = await transactionEntityManager.findOne(Purse, toUser.defaultPurseId); // <--there
const modalSum = Math.abs(sum);
if (fromPurse.balance < modalSum) {
throw new Error(NOT_ENOUGH_MONEY(fromId));
}
fromPurse.balance -= sum;
toPurse.balance += sum;
await this.appServiceV2.savePurse(fromPurse); // <-- oops, something was wrong
if (withError) {
throw new Error('Unexpectable error was thrown while remittance');
}
await transactionEntityManager.save(toPurse);
const remittance = new RemittanceResultDto();
remittance.fromId = fromId;
remittance.toId = toId;
remittance.fromBalance = fromPurse.balance;
remittance.sum = sum;
return remittance;
}
The whole project is synthetic, but to show the unpleasantness of this approach - I moved the
savePurse
method used to save the wallet into a separate appServiceV2
service
, and used this service with this method inside the considered makeRemittanceWithTypeOrmV1
method. You can see the code of this method and service below:@Injectable()
export class AppServiceV2 {
constructor(
@InjectRepository(Purse)
private readonly purseRepository: Repository<Purse>,
) {
}
async savePurse(purse: Purse) {
await this.purseRepository.save(purse);
}
// ...
}
Actually, in this situation, we get the following SQL queries:
START TRANSACTION
// ...
SELECT "User"."id" AS "User_id", "User"."name" AS "User_name", "User"."defaultPurseId" AS "User_defaultPurseId"
FROM "user" "User"
WHERE "User"."id" IN ($1)
START TRANSACTION // <-- this transaction from appServiceV2 😩
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
If we send a request for an error to occur, we will clearly see that the internal transaction from
appServiceV2
is not rolled back, and therefore our users are indignant again.START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
START TRANSACTION
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
ROLLBACK
Here we conclude that for a standard approach to trunking, you need to have special methods into which you will need to pass
transactionEntityManager
.If we want to get rid of the need to explicitly inject the
transactionEntityManager
into the corresponding methods, then the documentation advises us to look at decorators.By applying them, we get this kind of controller action:
// ...
@Post('remittance-with-typeorm-transaction-decorators')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTypeOrmTransactionDecorators(@Body() remittanceDto: RemittanceDto) {
return this.appService.makeRemittanceWithTypeOrmV2(remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
}
// ...
Now it has become simpler - there is no need to use the
connection
class, neither in the constructor, nor by calling the global method TypeORM. Perfectly. But the method of our service should still receive a dependency - transactionEntityManager
. This is where those decorators come to the rescue:// ...
@Transaction() // <-- this
async makeRemittanceWithTypeOrmV2(fromId: number, toId: number, sum: number, withError: boolean, @TransactionManager() transactionEntityManager: EntityManager = null /* <-- and this */) {
const fromUser = await transactionEntityManager.findOne(User, fromId);
const toUser = await transactionEntityManager.findOne(User, toId);
if (fromUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(fromId));
}
if (toUser === undefined) {
throw new Error(NOT_FOUND_USER_WITH_ID(toId));
}
if (fromUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(fromId));
}
if (toUser.defaultPurseId === null) {
throw new Error(USER_DOES_NOT_HAVE_PURSE(toId));
}
const fromPurse = await transactionEntityManager.findOne(Purse, fromUser.defaultPurseId);
const toPurse = await transactionEntityManager.findOne(Purse, toUser.defaultPurseId);
const modalSum = Math.abs(sum);
if (fromPurse.balance < modalSum) {
throw new Error(NOT_ENOUGH_MONEY(fromId));
}
fromPurse.balance -= sum;
toPurse.balance += sum;
await this.appServiceV2.savePurseInTransaction(fromPurse, transactionEntityManager); // <-- we will check is it will working
if (withError) {
throw new Error('Unexpectable error was thrown while remittance');
}
await transactionEntityManager.save(toPurse);
const remittance = new RemittanceResultDto();
remittance.fromId = fromId;
remittance.toId = toId;
remittance.fromBalance = fromPurse.balance;
remittance.sum = sum;
return remittance;
}
// ...
We have already figured out the fact that simply using a third-party service method breaks our transactions. Therefore, we used the new method of the third-party service
transactionEntityManager
, which looks like this:// ..
@Transaction()
async savePurseInTransaction(purse: Purse, @TransactionManager() transactionManager: EntityManager = null) {
await transactionManager.save(Purse, purse);
}
// ...
As you can see from the code, in this method we also used decorators - this way we achieve uniformity across all methods in the project (yep yep), and also get rid of the need to use
connection
in the constructor of controllers using our service appServiceV2
.With this approach, we get the following requests:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
START TRANSACTION
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
And, as a consequence, the destruction of the transaction and application logic on error:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
START TRANSACTION
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
ROLLBACK
The only working way, which the documentation describes, is to avoid using decorators. If you use decorators in all methods at once, then those of them that will be used by other services will inject their own
transactionEntityManagers
, as happened with our appServiceV2
service and its savePurseInTransaction
method. Let's try to replace this method with another:// app.service.ts
@Transaction()
async makeRemittanceWithTypeOrmV2(fromId: number, toId: number, sum: number, withError: boolean, @TransactionManager() transactionEntityManager: EntityManager = null) {
// ...
await this.appServiceV2.savePurseInTransactionV2(fromPurse, transactionEntityManager);
// ...
}
// app.service-v2.ts
// ..
async savePurseInTransactionV2(purse: Purse, transactionManager: EntityManager) {
await transactionManager.save(Purse, purse);
}
// ..
For the consistency of our methods, and getting rid of the hierarchy that has appeared, which is manifested in the fact that some methods can call others, but still others will not be able to call the first - we will change the method of the
appService
class. Thus, having received the first option from the documentation.Well, it seems we still have to inject this
connection
into the controller constructors. But the proposed way of writing code with transactions still looks very cumbersome and inconvenient. What to do?
Solving this problem, I made a package that allows you to use transactions in the simplest way. It is called nest-transact.
What is he doing? Everything is simple here. For our example with users and money transfers, let's look at the same logic written with nest-transact.
The code of our controller has not changed, and since we have made sure that we cannot do without
connection
in the constructor, we will specify it:@Controller()
@ApiTags('app')
export class AppController {
constructor(
private readonly appService: AppService,
private readonly connection: Connection, // <-- use this
) {
}
// ...
}
Controller's action:
// ...
@Post('remittance-with-transaction')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTransaction(@Body() remittanceDto: RemittanceDto) {
return await this.connection.transaction(transactionManager => {
return this.appService.withTransaction(transactionManager)/* <-- this is interesting new thing*/.makeRemittance(remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
});
}
// ...
Its difference from the action, in the case of using the first method from the documentation:
@Post('remittance-with-typeorm-transaction')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTypeOrmTransaction(@Body() remittanceDto: RemittanceDto) {
return await this.connection.transaction(transactionManager => {
return this.appService.makeRemittanceWithTypeOrmV1(transactionManager, remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
});
}
Is that we can use the usual methods of services without creating specific variations for transactions in which it is necessary to pass
transactionManager
. And also - that before using our service business method, we call the withTransaction
method on the same service, passing our transactionManager
to it. Here you can ask the question - where did this method come from?
Hence:
@Injectable()
export class AppService extends TransactionFor<AppService> /* <-- step 1 */ {
constructor(
@InjectRepository(User)
private readonly userRepository: Repository<User>,
@InjectRepository(Purse)
private readonly purseRepository: Repository<Purse>,
private readonly appServiceV2: AppServiceV2,
moduleRef: ModuleRef, // <-- step 2
) {
super(moduleRef);
}
// ...
}
And here is the request code:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
And with the error:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
ROLLBACK
But you already saw it at the very beginning.
To make this magic work, you need to complete two steps:
TransactionFor <ServiceType>
moduleRef: ModuleRef
in the list of constructor dependenciesIt's all. By the way, since dependency injection by the framework itself has not gone anywhere - you don't have to explicitly throw
moduleRef
. For testing only.You might be thinking - Why should I inherit from this class? What if my service will have to inherit from some other one? If you thought, then I suggest calculating how many of your services are inherited from other classes and are used in transactions.
Now how does it work? The appeared
withTransaction
method - recreates your service for this transaction, as well as all the dependencies of your service and the dependencies of dependencies - everything, everything, everything. It follows that if you somehow store some state in your services (but what if?) - then it will not be there when creating a transaction in this way. The original instance of your service still exists and when you call it, everything will be as before.In addition to the previous example, I also added a greedy method: transfer with commission, which uses two services at once in one controller action:
// ...
@Post('remittance-with-transaction-and-fee')
@ApiResponse({
type: RemittanceResultDto,
})
async makeRemittanceWithTransactionAndFee(@Body() remittanceDto: RemittanceDto) {
return this.connection.transaction(async manager => {
const transactionAppService = this.appService.withTransaction(manager); // <-- this is interesting new thing
const result = await transactionAppService.makeRemittance(remittanceDto.userIdFrom, remittanceDto.userIdTo, remittanceDto.sum, remittanceDto.withError);
result.fromBalance -= 1; // <-- transfer fee
const senderPurse = await transactionAppService.getPurse(remittanceDto.userIdFrom);
senderPurse.balance -= 1; // <-- transfer fee, for example of using several services in one transaction in controller
await this.appServiceV2.withTransaction(manager).savePurse(senderPurse);
return result;
});
}
// ...
This method makes the following requests:
START TRANSACTION
// ...
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
// this is new requests for fee:
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."userId" = $1
LIMIT 1
SELECT "Purse"."id" AS "Purse_id", "Purse"."balance" AS "Purse_balance", "Purse"."userId" AS "Purse_userId"
FROM "purse" "Purse"
WHERE "Purse"."id" IN ($1)
UPDATE "purse"
SET "balance" = $2
WHERE "id" IN ($1)
COMMIT
From which we can see that all requests still occur in one transaction and it will work correctly.
Summing up, I would like to say - when using this package in several real projects, I got a much more convenient way of writing transactions, of course, within the Nest.js + TypeORM stack. I hope you find it useful too. If you like this package and decide to give it a try, little wish - give it an asterisk on GitHub. It's not difficult for you, but it's useful for me and this package. I will also be glad to hear constructive criticism and possible ways to improve this solution.
Previously published here.