VANCOUVER, BC / ACCESSWIRE / November 1, 2023 / iMetal Resources Inc. (TSXV:IMR)(OTCQB:IMRFF)(FRANKFURT:A7V) (" iMetal " or the " Company ") announces an extension of the Digitally Enhanced

iMetal Resources Extends Digitally Enhanced Prospecting Survey on Its Gowganda West Project

SLT-MOBITEL, Sri Lanka’s National ICT Solutions Provider, has become the country’s first service provider to join the Oracle Partner Network (OPN), a global community designed to enable partners to accelerate their customers’ transition to the cloud and drive superior outcomes.

SLT-MOBITEL Sri Lanka’s first ICT service provider joins Oracle Partner Network

Knightscope, Inc. [Nasdaq: KSCP] ("Knightscope" or the "Company"), a leading developer of autonomous security robots and blue light emergency communication systems, today announces the expansion of an existing retail client and a new contract with a university in Alaska.

Retailer Expands and University Upgrades with Knightscope Technologies

Inceptio's L3/L2+ heavy-duty trucks represent major stride towards autonomous freight networks that will revolutionize line-haul logistics SHANGHAI, Oct. 31, 2023 /PRNewswire/ -- Inceptio Technology ("Inceptio,

Inceptio Executives Lay Out Vision for the Future of Freight Mobility at Major Global Tech Events

Caci International (CACI – Research Report), the Technology sector company, was revisited by a Wall Street analyst today. Analyst Cai von

Caci International’s Financial Outlook: Strong Performance and Promising Prospects Justify Buy Rating

Sanofi CEO Paul Hudson and Insight CEO Joyce Mullen say that for leaders who integrate AI mindfully, 'the resulting force will be unrivaled.'

Sanofi and Insight Enterprises CEOs: With the right blueprint in place, AI will empower workers and help big companies become more nimble

IMTE stock has gone up by 84.33%, with a monthly gain of 17.62% and a quarterly plunge of -10.83%. The volatility ratio for the week is 22.16%, and the volatility levels for the last 30 days are 15.83

Integrated Media Technology Limited’s (IMTE) Stock: A 61.03% Annual Performance Rate

A top tech company has chosen Immokalee as a place to inspire the next generation. The Immokalee Foundation is working with Microsoft to foster inclusive economic opportunity, job creation, and innovation in Southwest Florida.

Microsoft chooses Immokalee for National Pilot Program to teach students about tech

The initial release of KBC Acuity Industrial Cloud Suite comprises the Operations Monitoring family that targets upstream operations in the oil and gas industry. It delivers a wide range of functionalities for remote monitoring,

KBC Launches Acuity Industrial Cloud Suite for Digital Transformation

The Vidhance for Remote Assistance software suite uses Premium Video Stabilization and Click & Lock to improve viewer experiences and enable remote control of the video's focus point.

Imint Launches Powerful Video Stabilization Technology For Remote Assistance Devices

Maxilect

maxilect

Load Balancing Strategies for Applications: From Infrastructure to Code

The date of Cavalier’s departure drew near

Swoft is a PHP high performance microservice co-routine framework. It has been published for many years and has become the best choice for php. It can be like Go, built-in co-routine web server and common co-routine client and is resident in memory, independent of traditional PHP-FPM. There are similar Go language operations, similar to the Spring Cloud framework flexible annotations. Github https://github.com/swoft-cloud/swoft Feature Full Co-routine Framework Swoft is first PHP resident in memory annotation framework, with Spring Boot conventions larger than the configuration design concept, with a set of development specifications. AOP AOP is an object oriented programming that makes it easier to decouple business code, improve code quality, and increase code reusability. { $start; { ->start = microtime( );
    } {
        $method = $joinPoint->getMethod();
        $after = microtime( );
        $runtime = ($after - ->start) * ; ;
    }
} /**
 * (order=1)
 * (include={"App\Http\Controller\TestExecTimeController"})
 */ @Aspect @PointBean class CalcExecTimeAspect protected /**
     * ()
     */ @Before public function before () $this true /**
     * ()
     */ @After public function after (JoinPoint $joinPoint) true $this 1000 echo "{$method} cost: {$runtime}ms\n" Http Service Http service is simple and flexible, only to use @Controller() and @RequestMapping(route="index") annotations to define service. { { ;
    }
} /**
 * ()
 */ @Controller class IndexController /**
     * (route="index")
     */ @RequestMapping public : function index () string return "test" WebSocket service Swoft provides a complete Websocket for developers to quickly build service { {
        server()->push($request->getFd(), );
    }
} /**
 * (
 *     "/chat",
 *     messageParser=TokenTextParser::class,
 *     controllers={HomeController::class}
 * )
 */ @WsModule class ChatModule /**
     * ()
     * Request $request
     * int     $fd
     */ @OnOpen @param @param public : function onOpen (Request $request, int $fd) void "Opened, welcome!(FD: $fd)" RPC Service Swoft RPC can be called like a native function as Dubbo. { $userService; {
        $result  = ->userService->getList( , ); [$result];
    }
} /**
 * ()
 */ @Controller class RpcController /**
     * (pool="user.pool", version="1.0")
     *
     * UserInterface
     */ @Reference @var private /**
     * ("getList")
     *
     * array
     */ @RequestMapping @return public : function getList () array $this 12 'type' return TCP Service Swoft also provides feature-rich TCP service support. (strict_types= ); \ \ ; \ \ \ \ \ ; \ \ \ \ \ ; \ \ \ ; \ \ \ ; { { $str = $request->getPackage()->getDataString();

        $response->setData( . $str);
    }
} <?php declare 1 namespace App Tcp Controller use Swoft Tcp Server Annotation Mapping TcpController use Swoft Tcp Server Annotation Mapping TcpMapping use Swoft Tcp Server Request use Swoft Tcp Server Response /**
 * Class DemoController
 *
 * ()
 */ @TcpController class DemoController /**
     * ("echo", root=true)
     * Request  $request
     * Response $response
     */ @TcpMapping @param @param public : function echo (Request $request, Response $response) void // $str = $request->getRawData(); '[echo]hi, we received your message: ' Connection pool Swoft is simple to define a high-performance connection pool with like this: [ => [ => \Swoft\xxx\Pool::class, => , => , => , => , => ,
    ]
]; return 'xxx.pool' 'class' 'minActive' 10 'maxActive' 20 'maxWait' 0 'maxWaitTime' 0 'maxIdleTime' 60 Compatible with Laravel ORM Swoft database is highly compatible with Laravel ORM, It’s easy for PHP developer to use in Swoft. $user = User::new();
$user->setName( );
$user->setSex( );
$user->setDesc( );
$user->setAge(mt_rand( , ));
$user->save();
$id = $user->getId(); $users = DB::table( )->get(); ($users $user) { $user->name;
} DB::beginTransaction();
$user = User::find($id);
$user->update([ => $id]);

DB::beginTransaction();
User::find($id)->update([ => ]);
DB::rollBack();

DB::commit(); // Model 'name' 1 'this my desc' 1 100 // Query 'user' foreach as echo // Transaction 'name' 'name' 'sakuraovq' Microservice Swoft provides a set of quick build microservice governance components, it’s easy for developers to use. Service Registration and DiscoveryService BrokerCentralized ConfigurationService Throttling ability { { ( );
    } {
        $uri = $request->getUriPath(); [ , $uri];
    }
} /**
 * ()
 */ @Bean class Test /**
     * (fallback="funcFallback")
     *
     * string
     * Exception
     */ @Breaker @return @throws public : function func () string // Do something throw new Exception 'Breaker exception' /**
     * ()
     * (key="request.getUriPath()")
     *
     * Request $request
     *
     * array
     */ @RequestMapping @RateLimiter @param @return public : function requestLimiter (Request $request) array return 'requestLimiter' Github https://github.com/swoft-cloud/swoft

Evaluating Swoft — A PHP Microservices Framework

A few weeks ago, Simon Stone, Lead Engineer at the IBM team, announced that , leaving a lot of developers with existing applications in a cloudy situation . Blockchain IBM will significantly reduce its efforts in the Hyperledger Composer project wondering what to do next The good news, here at we have been working in an Smart Contract framework that , so you don't have to think about managing the complexities of communicating with Fabric directly, but still running a native Node JS chaincode on it. Additionally on top of it, we're building several tools to make the developer’s life easier, like a dev environment ready to be used out of the box, just like you used to work in . WorldSibu runs natively in Hyperledger Fabric Composer Right now Convector supports Hyperledger Fabric already, and we're planning to add support for multiple Blockchain techs soon, stay tuned! In this example, I'll show you how you can easily migrate from Composer to . This can be done without too much efforts since we share pretty similar concepts. Convector The first difference between the two is the language. Composer has decided to go with its custom designed language to define the different pieces of a smart contract. Convector on the other hand is crafted using , a super set language based on Javascript that has been gaining traction because of all the rich features it has, like optional strong typing and abstract syntax trees (AST) generation to create tools around the code. Typescript In Composer, a Business Network model is defined by essentially three components: : houses and listings Assets buyers and homeowners Participants: buying or selling houses, and creating and closing listings Transactions: When we were designing Convector we stoped here and thought: assets and participants are basically models compound by different properties, and even more, participants can sometimes act as the one making an action, or the one the action is being made on! We wanted to keep things as simple as possible, and give the developers the ability to express their ideas with a more software development friendly terminology, thus we decided to create two fundamental concepts inside the framework: A reference to in the real world, tangible or intangible, describing its properties. It might as well be a participant in the network. Models: something A set of rules on how you can interact with the models. Controllers: Based on those two fundamental concepts you can build everything you need for your chaincode and yet have enough flexibility to express your logic the way you need. The following piece of code defines the Marbles example in Composer. namespace org.hyperledger_composer.marbles enum MarbleColor {o REDo GREENo BLUEo PURPLEo ORANGE} enum MarbleSize {o SMALLo MEDIUMo LARGE} asset Marble identified by marbleId {o String marbleIdo MarbleSize sizeo MarbleColor color--> Player owner} participant Player identified by email {o String emailo String firstNameo String lastName} In Convector you write it this way: export enum MarbleColor {RED,GREEN,BLUE,PURPLE,ORANGE} export enum MarbleSize {SMALL,MEDIUM,LARGE} export class Marble extends ConvectorModel<Marble> {@ReadOnly()@Required()public type = 'io.worldsibu.marbles.marble'; @Validate(yup.number())@Default(MarbleSize.MEDIUM)public size: MarbleSize; @Validate(yup.number())public color: MarbleColor; @Validate(yup.string().email())public owner: string;} export class Player extends ConvectorModel<Player> {@Required()@ReadOnly()public type = 'io.worldsibu.marbles.player'; @Validate(yup.string().email())public id: string; @Validate(yup.string())public firstName: string; @Validate(yup.string())public lastName: string;} Since Convector runs on Javascript, you can have expressions for validations and restrict the data your model contains in a more expressive way. We have created a set of useful decorators to help you achieve this and save you some code while keeping the models readable (and reducing risks!). There are two properties a model must have, a which is a unique identifier for this model class and the which is an unique identifier for each model instance. If you don't specify and id field, Convector will declare a string field for you. type id Chaincode Logic In Composer the chaincode logic, a.k.a the rules that defines what you can do with the data, resides in 3 different layers: ACL, Transaction Models, and Transaction Functions. First, you start by defining the access permissions in the ACL (access control list): rule Default {description: "Allow all participants access to all resources"participant: "org.hyperledger_composer.marbles.player"operation: ALLresource: "org.hyperledger_composer.marbles.*"action: ALLOW} Then you define the transaction model: transaction TradeMarble {--> Marble marble--> Player newOwner} And finally you define the transaction function: async function tradeMarble(tradeMarble) {tradeMarble.marble.owner = tradeMarble.newOwner; const assetRegistry = await getAssetRegistry('org.hyperledger_composer.marbles.Marble'); await assetRegistry.update(tradeMarble.marble);} In we simplified all this logic using Convector Controllers @Controller('marble')export class MarbleController extends ConvectorController { @Invokable()public async create(@Param(Marble) marble: Marble) {await marble.save();} @Invokable()public async trade(@Param(yup.string()) marbleId: string,@Param(yup.string().email()) newOwner: string): Promise<void> {// use this.sender for authorization checks const marble = await Marble.getOne(marbleId);

marble.owner = newOwner;

await marble.save(); }} You can define the transaction model using the and decorators. Any Invokable method will be exposed as a transaction, and the Params describe its signature. You're not restricted to primitives only, you can pass as well, Convector will parse all complex objects for you. @Invokable @Param Models You can define the access rights using , which is the sender’s fingerprint, unique per participant. You can do all sort of things with it, even attach multiple senders to a single participant if you want. This is the flexibility it provides so you can define your own rules. this.sender And finally, you write all the necessary extra logic for the transaction function. You can do almost all kind of operations in here, as long as they are deterministic and use no external data other than the ledger data. For querying the ledger, every model has some useful CRUD methods like or , , and . There are some others like , which retrieves the history of updates for that model, to duplicate an asset or to batch assign properties to the model. getOne getAll save update delete history clone assign Client Access If you want to test the above methods in Composer, this is what you usually do: const namespace = 'org.hyperledger_composer.marbles';const factory = businessNetworkConnection.getBusinessNetwork().getFactory(); const dan = factory.newResource(namespace, 'Player', 'daniel.selman@example.com');dan.firstName = 'Dan';dan.lastName = 'Selman'; const simon = factory.newResource(namespace, 'Player', 'sstone1@example.com');simon.firstName = 'Simon';simon.lastName = 'Stone'; const playerRegistry = await businessNetworkConnection.getParticipantRegistry(namespace + '.Player');await playerRegistry.addAll([dan, simon]); const marble = factory.newResource(namespace, 'Marble', 'MARBLE_001');marble.size = 'SMALL';marble.color = 'RED';marble.owner = factory.newRelationship(namespace, 'Player', dan.$identifier); const marbleRegistry = await businessNetworkConnection.getAssetRegistry(namespace + '.Marble');await marbleRegistry.add(marble); const tradeMarble = factory.newTransaction(namespace, 'TradeMarble');tradeMarble.newOwner = factory.newRelationship(namespace, 'Player', simon.$identifier);tradeMarble.marble = factory.newRelationship(namespace, 'Marble', marble.$identifier);await businessNetworkConnection.submitTransaction(tradeMarble); In Convector, we have adapters to be able to use the same controller and model in the client applications. const adapter = new FabricControllerAdapter(configuration);const playerCtrl = new PlayerControllerClient(adapter);const marbleCtrl = new MarbleControllerClient(adapter); const dan = new Player({id: 'daniel.selman@example.com',firstName: 'Dan',lastName: 'Selman'});await playerCtrl.register(dan); const simon = new Player({id: 'sstone1@example.com',firstName: 'Simon',lastName: 'Stone'});await playerCtrl.register(simon); const marble = new Marble({id: '1',size: MarbleSize.SMALL,color: MarbleColor.RED,owner: dan.id});await marbleCtrl.create(marble); await marbleCtrl.trade(marble.id, simon.id); Each adapter has its own internal logic, the for instance, handles the network profile for communication with Fabric, but Convector is not tightly coupled with Fabric, so the community can create different adapters based on their needs. This is exactly how we plan to support other blockchains in the near future, by replacing the adapter with a new implementation.Adapters don't have any special treatment inside convector, they just dictate the communication with the blockchain layer so that the standard adapters that we have in Convector can be replaced by a custom implementation provided by the developers or the community. FabricControllerAdapter Conclusion Composer and Convector share a lot of similarities, yet they took different architectural paths. With the Convector controllers you pretty much are in control of all the logic inside the chaincode and have much more flexibility to design architectural decision you consider are appropriate for your use case. We're constantly adding new features to the framework and that might be useful for the community. You can create new issues or even at any time. we'll love to hear from you about new features drop a PR in our github Join the community at Discord! Get the code!

Migrating from Hyperledger Composer to Convector Framework — Marbles Example

You must read this before releasing your Golang API into production. Based on our true story at Kurio, how we struggling for every release because we are not doing it in the right ways. laptop on fire taken from google image search A few weeks ago, we are in just fixing our weird and undetected bug in our main services. We have tried so many ways to debugging and fixing it. The bug is not with the business logic. Because it has been running on production for a few weeks. But we always saved by our auto-scaling mechanism, so it’s just like running well. Kurio Until later, we figured out it, it is because our code, we are not doing it well. Architecture Fyi, we are using a microservices pattern in our architecture. We have one gateway API — we call it — that serve API to our user (mobile, and web). Because of its role similar to API gateway, so its task is only handled request from the user, then call the required services, and build the response to user. This , written fully in Golang. The reason to choose golang is another story that I won’t tell it here. main API main API If drawn in a picture, our system will look more like this. Kurio architecture The Problem We have struggled for a long time with our main API, that always down and returning a long response to our mobile-apps and sometimes causing our API can’t be accessed. Our API dashboard monitor just turns to red — to be honest, when our API dashboard monitor goes red, it was a dangerous thing and causing stress and panic and crazy to us the engineer 😈. Other things are, our CPU and memory usage are getting higher. If this happens we just restart it manually and wait until it runs again. Our API response time up-to 86 seconds for a single request. graph our API response time, and doing restart manually for safety. This bug really makes us frustrated, because we don’t have any log that specifically telling about this bug. We just have that the response time is so long. The CPU and memory usage increasing. It just like a nightmare. Phase 1 : Using Customized http.Client One thing we learn and really really learn when developing this service is, don’t trust the default config. We use a customized , instead using the default one from the http’s package, http.Client client:=http.Client{} //default We add some config based on our need. Because we need to reuse the connection, we make some configuration in our transport and control max-idle reusable connection. keepAliveTimeout:= 600 * time.Secondtimeout:= 2 * time.SeconddefaultTransport := &http.Transport{Dial: (&net.Dialer{KeepAlive: keepAliveTimeout,}).Dial,MaxIdleConns: 100,MaxIdleConnsPerHost: 100, } client:= &http.Client{Transport: defaultTransport,Timeout:   timeout,} This configuration can help us to reduce the maximum of time to be used to calling another service. Phase 2: Avoiding Memory Leak From Unclosed Response’s Body What we learn from this phase is: If we want to reuse our connection pool to another service, we must read the response body, and close it. Because our is just calling another service, we make a fatal mistake. Our suppose to reuse the available connection from so whatever happens, we must read the response body, even we don’t need it. And also we must close the response body. Both of this used to avoid memory leak in our server. main API main API http.Client We forgot to close our response body in our code. And this things can cause a huge disaster for our production. The solution is: we close the response body and read it even we don’t need the data. req, err:= http.NewRequest("GET","http://example.com?q=one",nil)if err != nil {return err} resp, err:= client.Do(req)//=================================================// CLOSE THE RESPONSE BODY//=================================================if resp != nil {defer resp.Body.Close() // MUST CLOSED THIS}if err != nil {return err} //=================================================// READ THE BODY EVEN THE DATA IS NOT IMPORTANT// THIS MUST TO DO, TO AVOID MEMORY LEAK WHEN REUSING HTTP// CONNECTION//=================================================_, err = io.Copy(ioutil.Discard, resp.Body) // WE READ THE BODY if err != nil {return err} We fix this after reading a great article here: and here: http://devs.cloudimmunity.com/gotchas-and-common-mistakes-in-go-golang/ http://tleyden.github.io/blog/2016/11/21/tuning-the-go-http-client-library-for-load-testing/ Phase 1 and Phase 2 and with the help of auto scaling success to reduce this bug. Well, to be honest, it never happens anymore for even 3 months since the last year: 2017. Phase 3: Timeout Control In Golang Channel After a few months run well, this bug happens again. At the first week of January 2018, one of our service that called by our , let says: is down. For some reasons, it cannot be accessed. main API So when our is down, our is going fired again. The API dashboard going red again, the API response time is going higher and slower. Our CPU and memory usage going very high even using the autoscaling. content service main API Again, we trying to find the root problem again. Well, after re-running the we run well again. content service For that’s case, we are curious, why this happens. Because we think, we have set the timeout deadline in , so in that case, this will never happen. http.Client Searching for the potential issue in our code, then we found some dangerous code. For more simple, the code more look like this * ps: this function is just an example, but similar in pattern with ours our code before fixed If we look the above code, it’s like nothing wrong. But this function is the most accessed, and have the heaviest call in our . Because this function will do 3 API call with huge processing. main API To improve this we make a new approachment using timeout-control on the channel. Because with the above style code — which using will wait until all the process done — we must wait for all the API call must finish so we can process and return the response to the user. WaitGroup This is one of our big mistakes. This code can make a huge disaster when one of our services died. Because there will be a long waiting until the died service recovered. With 5K call/s, of course, this is a disaster. First attempt solution: We modify it with adding a timeout. So our user will not wait for so long, they will just get an internal server error. trying to add timeout Phase 4: Timeout Control Using Context After doing our problem still not fully solved. Our still consume high CPU and memory. Phase 3, main API This happens because, even we already returning to our user, but our goroutine has still existed. What we want is, if we already return the response, then all the resource is also cleared, no exception, including goroutine and API call that running in the background. Internal Server Error Later after reading this article: http://dahernan.github.io/2015/02/04/context-and-cancellation-of-goroutines/ We found some interesting feature that we do not realize yet in golang. That was using to help cancellation in go-routine. Context Instead of using to use the timeout, we move to . With this new approachment, our service is more reliable. time.After context.Context Then we change our code structure again by adding context to function that related. final fixed code. using context So then we use for every goroutine call in our code. This helps us to release memory and cancel the goroutine call. Context In addition, for more controlled and reliable, we also pass the context to our HTTP request. using context on HTTP request With all of this setting and timeout control, our system is more safe and controllable. Lesson learned: Never used the default option in production.Just never used the default option. If you’re building a big concurrent A, just never use the default option. Read a lot, tried a lot, failed a lot, and gain a lot.We learn a lot from this experience, this experience only gained in real cases and real user. And I’m so happy being part of it when fixing this bug. * last updated: 18 January 2018: fixing some of the typos If you think this worth enough to read, give me a 👏 also share this on your twitter, facebook etc, so other people can read this. If you have a question or something, you could put a response below or just email me.

Avoiding Memory Leak in Golang API

A story of how we develop the targeted push notification service from backend-side at Kurio. A few weeks ago, we in , are developing a new feature. It’s a recommended/targeted push notification. It’s different from our current push notification system in which the push notification is sent to all users based on the data we had. With this feature, every user may get a different push notification. Kurio Our tools Go (1.8) Mysql Google Coud PubSub Cloud Messaging (GCM,APNS) How it works Our old system if drawn in picture could be like this below: This is how the normal push notification works. We chose an article to push based on the data we had about that article, then initiate a push notification from the dashboard, which will trigger the “push management” service. Each user will get the same push notification based-on the push. Then we decided to improve it by sending a targeted push, so that each user may get a different push notification based on what they previously read. The process will look like : Kurio Targeted Push Notification This new system looks more complex , because it uses an external service to provide the and based on users’ read.Since it takes some time to process all the users to get a targeted push notification, and the data would be so big, we decided to use Google Cloud PubSub. By using PubSub, we can start sending notification right after we process each user. user_id article_id Challenges The challenge we had when we decided on targeted push notification is to integrate with the external targeting system. The external targeting system might generate the tuple of and on-demand. And that targeting system might take some time to do some processing. Then we need to define how we want to consume that data as it might be large. user_id article_id Listening from Pubsub Machine With the challenges in mind, we decided to use Google Cloud PubSub.So that the external targeting system may take its time and just publish the tuple of and whenever its ready. Then we only need a subscriber to listen to that topic. user_id article_id Our worker then gets the necessary data to build the notification. The data we need, on this case, is the detail of the article (title, thumbnail, etc.) and user’s device info (push token). Thanks to our microservice architecture, we can get the article’s detail from a service that managed that data. And for the user’s device info, we already have it on our own database. First try So after we finish the code, we try the system in the production environment. For a few hundreds of users the system still running well. After expanding the number of users, the disaster happens. We got error. This happens as we spawn a goroutine for each message. As the number of goroutine increases, the number of connection to Mysql increases and it finally hits the limit. Too Many Connection Creating Connection Pooling After realizing the error, then we set a maximum connection pooling to database from golang itself. We set the maximum Open Connection and maximum Idle Connection. Read here: http://go-database-sql.org/connection-pool.html Kurio Connection Pooling dbConn.SetConnMaxLifetime(1 * time.Minute)dbConn.SetMaxIdleConns(10)dbConn.SetMaxOpenConns(250) When setting this, the things we must know is not to set the maximum open connection bigger then the Mysql could handle. In our instance, our database could handle up to 300 connections fine. Second Try After setting the connection pooling, now we handle the error for . But then we realize something weird happen. Our message queue already empties in the pubsub machine, but our worker still running processing data. And it took a long process. After finding the cause, we know that library we used in golang has the default Too Many Connections Maximum Outstanding Message : 1000 This outstanding message means, our projects accept message up to 1000 messages. This explains why the Subscription already emptied but our worker still runs for a long time. Set the MaxOutstandingMessage To reduce the queue in the worker, we figured that our outstanding message is too large than the connection to the database. And it makes our go routine grow and grow larger. Because we only test using around 60K messages, the goroutine might still be able to handle since goroutine is small and cheap in resources. But what happens next if we increase the number of users significantly? Before enhanced Max Outstanding Message in Pubsub Client To handle this, we set the maximum outstanding message that could be processed by our push worker. By doing so, our queue doesn’t grow too large as the memory footprint will grow as well. Reduce the Max Outstanding Message Because the messages were coming in faster than querying to DB for getting the active devices based on each user, we reduce the maximum outstanding message to 100, 10% from the default. This helps us to reduce the number of goroutine running in our worker. Third Try After making those improvements, we then tried again. It’s running well now that we optimized stuff. But then there are other problems a new and an unexpected one. After running with the improvements, we found another issue. Our system is push management service, and therefore, besides sending a push notification to GCM/APNS, we still have an API that serves a list of past notifications on the API side. This service is being used by both our dashboard and mobile app. The issue is that when our worker is sending the targeted push, we couldn’t access that API. It would just keep waiting until it finally timeout. We started to being confused about what causes it. After some investigations, we finally realized that it only happens when the worker is sending the targeted push notification. It turns out the workers have used up all the connection pool to Mysql and left the API waiting for a connection from the pool. Separate Mysql Client To handle that issue, with the delivery target in mind (a.k.a. deadline), we then decided to separate the Mysql client for the API and worker so that they both get their each connection pool independently. So now we have 2 Mysql client in a single project Create 2 Mysql Client Connection Instead, we use a single client to used by the entire system, we separate into 2 clients to the database. Client 1 will be used by the API system for the serving dashboard and mobile apps. And Client 2 will be used by the push worker. We then set the maximum connection in the API side lower than the worker side. Since the worker side needs a single connection for each user and we need to send it fast, it will need a lot more connections in the pool. With this setup, the dashboard and mobile app can access the API even when we’re sending the targeted push notification. Fourth Try Then for the fourth time, we try again. Running the targeted push notification. Now we can call the API push management and the dashboard run well even we are doing targeted push notification. But…. We figured that there are some weird behaviors. It’s not about the logic. Everything was running well right now. But for a single notification, it could take up to 3 hours for sending a message to cloud messaging. It’s really really so long. After looking at the logs, we can see that the query for getting device info per user is taking so long. WTF!!!! Improving the query: Using Index Then we started to investigate the query. Turns out that a single query took at least 30 seconds. Really? The problem wasn’t because we don’t have an index. We already have an index in our table. But it’s like the Mysql not use the proper index, but also it’s because of our slow query. SELECT `token` FROM device  WHERE `id` IN ( SELECT max(`id`) as `id` FROM device WHERE `deleted_at` IS NULL AND `user_id` IN (" ")  GROUP BY uuid ) userIDs To improve the query, we wrote a new query. Because we only need one latest device, so we changed it to: SELECT `token`FROM device (user_id)WHERE `user_id` = ?AND `deleted_at` IS NULLORDER BY id DESCLIMIT 1 USE INDEX With this query, we forced the Mysql to use the proper Index. And it saved us a lot of time for queries. With the new query, it now only took about 30 milliseconds Hooorayyy !!!! More speed, doing Prepare Statement Even after getting the speed of query we wanted, we also make some of the additional improvements. Kurio Prepared Statement Querying Instead of redoing the bunch of procedures every time we query (open connection, create a query, execute the query), we simplify it by using the prepared statement. We prepared the statement on the application started. Then inject the statement into a function that will be called by push worker. In the main function, we open the connection and prepare the statement dsn := "kurio:someofpassword@tcp(127.0.0.1:3306)/push_notification?parseTime=1&loc=Asia%2FJakarta" dbConn, _ := sql.Open(`mysql`, dsn) , err:= dbConn. (queryGetDeviceByUserID) StmtGetDevicesByUserID Prepare // stmtGetDevicesByUserID : will injected to function GetDeviceByUserID After prepared, the statement will injected to the function for retrieving the device. Then in that function, we just execute the prepared statement. StmtGetDevicesByUserID func (repo *Repository)GetDeviceByUserID(userID int64) *Device{ row:= repo. .QueryRow(userID) **StmtGetDevicesByUserID** device:= &Device{}err:= row.Scan(&device.Token)return device} So when each worker got a new message from Google Pubsub, it now only pass the user ID. Because the query and the boilerplate have already been prepared. Caching and HTTP Configuration For the HTTP call to Article service, we use some caching mechanism because there might be some users that get the same article. So to reduce the HTTP call to our article service, we added in-memory cache that will be expired for a certain of times (10 minutes for instances). We used an open-sources in-memory golang cache here https://github.com/patrickmn/go-cache And also make some configuration in the HTTP default transport timeout := 2 * time.SecondkeepAliveTimeout := 100 * time.SeconddefaultTransport := &http.Transport{Dial: (&net.Dialer{KeepAlive: keepAliveTimeout,}).Dial,MaxIdleConns:100,MaxIdleConnsPerHost:600,} client:= &http.Client{Transport: defaultTransport,Timeout:   timeout,} This is used for handling the HTTP call and timeout gracefully. The more explanation about this you could read in this good blog post by cloudfare here: https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/ Conclusions After a few long weeks developing this feature, there are many things I learned and many things that can help me improve my code Go routine is small and cheap, but because of their cheapness, we must carefully to take care of them, especially when dealing with connections and etc. Currently, we separate 2 clients because of the deadline, as our company wanted to do some initial tests regarding the targeting system. For now, we don’t have any issue about this. If something unexpected happens by this, we will make changes later, and update it here. If you think you can improve this for more better optimized, just say hi to 😄 join@kurio.co.id Thanks to , , and All Teams involved at . Andrew Ongko Rifad Ainun Nazieb Arie Ardaya Lizuardi Kurio If you think this worth enough to read, just share this on your twitter, facebook etc, so other people can read this. If you have a question or something, you could put a response below or just email me _Engineering-related notes from your friendly neighborhood, Kurio_toolbox.kurio.co.id Kurio Toolbox _Guide to make gRPC service using clean architecture in golang._toolbox.kurio.co.id Implementing gRPC service in Golang _Independent, Testable , and Clean_hackernoon.com Trying Clean Architecture on Golang

Developing Backend for Targeted Push Notification

Automatic Brand Sentiment Analysis with Generative AI

When I look at the whole startup ecosystem right now, it seems like money is flying around in massive amounts. Investors are raising bigger and bigger funds in ever shorter intervals, and you could spend your whole day reading funding announcements on Techcrunch, LinkedIn, or other Social Media outlets. It almost feels like everyone will get funded despite their traction or idea itself. But working in the ecosystem for quite some time, I can tell you something. Fundraising is hard. It was, still is, and also will be in the future. Only a tiny percentage of the startups looking for capital will get it and try to make their dream happen. Looking at the market size (in H1 of 2021, over $288 billion has been invested globally into uprising companies), it is not surprising that many companies try to build tools and resources for this market. And that's also what I love to do—supporting founders. But in this giant black box, it is tough to navigate and understand what tools you can choose from. That's why I decided to create the "Startup Fundraising Market Map" - a graphic overview of all companies in the market. Let’s jump into the categories: Pitch Decks Your pitch deck is one of the most important things in your fundraising process. It is a presentation that you sent out to potential investors that gives a short overview of your whole business. Investors get thousands of pitch decks every year, so you have to make sure to really stand out. In the map, you find companies that help you create this pitch deck through their tool, templates, or services. Furthermore, I added two websites that have collected you can get some inspiration from. hundreds of real pitch decks Alternative Funding Most funding options you will have as a startup are dilutive. This means you change equity in your company for money. But over the last years, more and more companies have been founded that are providing non-dilutive funding options. Using them, you are exchanging future revenues for money that you get now to grow your business. These providers are especially interesting if you already have a working product with customers. Financial Model Having a solid financial plan for the next years of your business is essential. The model helps you put your assumptions into writing but also plan the people you want to hire and the investment it needs. In general, you can either use one of the tools for financial planning our doing it yourself or with a financial model template in a spreadsheet program. Documents Before you get the money into your bank account, many documents will be sent back and forth to your investor. We collected the tools that you can use as an alternative to sending your pitch deck via email, the tools you need to sign investment documents, and the companies you can use to build a virtual data room for due diligence. Investors Of course, the people with the money must not be left out. First of all, there are companies that help you find investors by providing their data in lists or through a startup<>investor matching. Secondly, there are platforms that help startups and investors set up deal structures and entities. Because I strongly believe that there has to be a startup investor fit, I also included companies building a review layer for investors. Because in the end, you have to work with them for a long time. So better make sure you are a fit. Education & Communities Founders should really support each other, and I took so much from experiences shared in my career. That’s why I collected some of the best founder communities and education pages. IRM & CRM I always tell the startups I am advising that one of the most important things in your fundraising is to run a structured process. An investor relationship management system helps you track all your interactions with potential investors and update the ones you already have. Captable & Equity Management Getting your company structure and equity right is a very time-consuming task. A new breed of tools makes it easy to do that and incentivize your employees. Data As a founder, you need to understand the market you are in. There are several platforms that can help you in your research. I hope you liked the map. If you want to have more content on Startup Fundraising make sure to follow me on . Twitter All links & .pdf can be downloaded at https://www.basetemplates.com/startup-fundraising-market-map

Stories, News & Mentions about MAXILECT

Maxilect’s Stories on HackerNoon

Maxilect's News & Mentions