Performance testing server side rendering using java template engines + Spring Boot and some learnings on the way This article is about benchmarking server side rendering tools i.e the most commonly used based templating engines i.e , and (new to me) and lessons I learnt while doing so. I have already worked with Velocity and Freemarker as my server side render tools but after reading about HTTL, I wanted to get real insights of how actually these 3 will perform with a project. java Velocity Freemarker HTTL Spring Boot Below is the system info where java server side render/template benchmarking will happen: OS: MacOS Sierra Processor: 1.6 GHz Intel Core i5 Java: Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode) For benchmarking we are using . The code is a simple spring boot app exposing three endpoints and three spring based view resolvers like below snippet. Apache bench You can download the complete code from this github repo @SpringBootApplication@Controllerpublic class TemplateBenchmark { @Bean public HttlViewResolver httlViewResolver() { HttlViewResolver resolver = new HttlViewResolver(); … return resolver; } @Bean public ViewResolver vmViewResolver() { VelocityLayoutViewResolver bean = new VelocityLayoutViewResolver(); …. return bean; } @RequestMapping(“/checkout-httl”) public String checkout(Model model) { model.addAttribute(“name”, “Apurav Chauhan”); return “checkout-httl”; }} ROUND 1 — Simple key val + HTML render HTTL Template engine for server side rendering We warm up the server with 5000 requests in total with a concurrency of 5. Finally, we set the bench to hit 25K requests with a concurrency of 25 using the following command ab -n 25000 -c 25 -g httl.tsv -k http://localhost:9090/checkout-httl -n flag for total requests | -c for concurrency | -g to save bench stats to a tab separated file | -k to enable keep alive connections Learning 1 While doing benchmarking, if you see this error its because load has consumed all the available 16,383 ports on MacOS. ”apr_socket_recv: Operation timed out (60)” As each port is used, it get’s put into a queue where it waits for the tcp “Maximum Segment Lifetime”, which is configured to be 15 seconds on osx. So if you use more than 16,383 ports in 15 seconds, you’re effectively going to get throttled by the OS on further connections. Depending on which process runs out of ports first, you will get connection errors from the server, or hangs from [_ab_](https://stackoverflow.com/a/30357879/949912) . To fix this, use the below command to check the default wait time in your macos terminal. sysctl net.inet.tcp.msl And update it to a lower value like 1000ms by using following command sudo sysctl -w net.inet.tcp.msl=1000 And now here are the stats: HTTL VisualVM Stats . CPU usage bumped up to 70%. Mean processing time for a single request was 0.134ms A total of 25K requests were completed within ~3secs Freemarker Template Engine for java server side render The same step was repeated to fetch the below stats FTL JVM Stats . CPU usage bumped up to 30%. Mean processing time for a single request was 0.476ms A total of 25K requests were completed within ~12secs Velocity Template Engine for java server side rendering VM JVM Stats . CPU usage bumped up to 90%. Mean processing time for a single request was 0.253ms A total of 25K requests were completed within ~6secs GNUPLOT FOR BENCHMARK DATA Using the tsv file that was generated via -g flag, we will now generate a graph to actually see how each of these engines performed in comparison to each other. You can check the template files in and github links that were used to create the below graphs. Run the below command : this this gnuplot template.p gnuplot template2.p Benchmarks HTTL vs Freemarker vs Velocity HTTL definitely shines, but Freemarker shows a huge downside. Lets dig deeper. While analyzing the ab results, we can see that freemarker didn’t use any keep alive requests causing the skewed behaviour by including the connection creation lag in the results. But why? Learning 2 After some research, found that ab tool works on Http/1.0 protocol that defines the server to send a Content-Length header to respect the keep alive connection. Freemarker responds with a chunked encoded response thus NOT allowing the connection reuse using Connection: KeepAlive header. Lets update the code to set content length header for our simple use case and see what we get: Seems like Freemarker is back in the race! :) Still not satisfied? Lets try to benchmark the three using now. I am using 25 users with a ramp-up period of 20 secs with a loop count of 25k and below are the results: jmeter Jmeter stats for response time Performance of HTTL and Freemarker seems to be comparable for a simple template render with a key value object. However, Velocity is completely out of the picture! Here is the used to benchmark java server side templates discussed above. Jmeter test plan Round 2 — Complex object + HTML rendering Coming soon
