At , our unique approach to contextualizing real-world events and actions with academic concepts makes Maths, Science and English highly engaging for children! Our engaging explanations and fun assessments, delivered through , Android and iOS, make heavy use of static images, animated images, and videos. Genius Progressive Web Apps (PWA) A Genius way of learning Need of the Hour There was a time when the users were patient enough to wait for the media to load. Image loading time of 0.5 sec or more is quite unacceptable now. A jagged experience doesn’t keep users happy, defeating the purpose of great products. Though the internet connectivity across the world is improving significantly, latency and bandwidth fluctuations are . still quite common A typical Joe, not impressed with your service This article will help you prepare for the worst and get really close to making your media available as fast as possible. A great fluid experience is provided by optimizing across 3 main factors: during asset download. User Experience Reducing download time. for near instant loading. Browser Caching There are many articles which help with one or the other factors. This article gives an almost complete summary of ideas with a solution to run smooth media-rich web apps on spotty networks. It is intended for users who understand fundamentals of — HTML, CSS, HTTP requests and common browser behaviour. web development At Genius, we heavily use Typescript, React and Node.js. Our PWAs, internal dashboards, tools and server apps are all built with them. Naturally, the solutions are also presented in our field of expertise. User Experience Keep users engaged. Show them something which blends into what is about to come. Here are few ways we deploy at Genius to make our experience great. Animated loading area A simple rectangle, animating with 2 colours, goes a long way in providing a psychological feeling of “something is happening”. It creates an event which registers subconsciously, and tricks it into believing, “Let me wait and see what shows up.” An animating box of subconscious trickery A simple feature like this is quite easy to pull off using plain CSS. First, define a set of keyframes for the background colour. loading {0% { background-color: #e0e0e0; }50% { background-color: #eeeeee; }100% { background-color: #e0e0e0; }} @keyframes Animate them in a CSS class. .img-container {background-color: #ffffff;animation: loading 1.3s ease-in-out infinite;-webkit-animation: loading 1.3s ease-in-out infinite;} Use the class in the image container which will have the tag. <img> <div class="img-container" style="height: 500px; width: 650px;"><img src="stove.jpg"></div> Progressive Loading There are 2 types of JPEG — and . Progressive shows the complete, low-quality image first, and improving the precision as the data arrives. It also leads to smaller images (detail below). Baseline Progressive There are many to build progressive images. At Genius, we have built a Node.js package (written in Typescript) for parsing and packaging our content and media. It uses to resize and make all JPEG progressive. online tools Lovell Fuller’s Sharp Does progressive JPEG result in a better experience? The answer is not definitive. It probably leads to more engagement but that it leads to decreased user happiness because of the increased cognitive load. research suggests At Genius, we ended up using progressive to keep the users engaged and to keep the size of the JPEG minimum (more on this below). Poster Image for Videos The of a video is the first chance to engage a user while the video downloads. From point of view, it is a bad idea to not put a poster. It helps in keeping the user engaged subconsciously. poster image user experience A poster image is provided through the attribute on the video element . poster <video> <video poster="poster.jpg" controls><source src="movie.mp4" type="video/mp4"></video> Reducing Download Time To improve the download time of any resource, we can optimize on 3 main factors: : downsize the resolution of images and videos, use correct encoding format to bring down the size even more. Size of the asset : HTTP/2 leaves HTTP/1.x in dust. HTTP protocol : unfinished entries in the request queue, which are no longer needed, increase the loading time of the assets which are needed. Request queue Downsized Assets The best case scenario is to deliver the smallest resolution image/video with the best possible format and compression while maintaining acceptable quality. It is commonly understood to NOT deliver high-resolution, high-quality images unless you are a “photograph shop” where quality matters. There are primarily two ways of doing it: Client-side HTML to let the browser decide from a set of images/videos Server sends the exact resized image with resolution specified in the URL Client-Side HTML Using with HTML and tags makes it quite simple to let the browser decide which image to load. All you have to do is create multiple version of images and save on the server. Here is an example: CSS media queries [<picture>](https://www.w3schools.com/tags/tag_picture.asp) [<source>](https://www.w3schools.com/tags/tag_source.asp) <picture><source media="(min-width: 451px)" srcset="picture-660x414.jpg"><source media="(max-width: 450px)" srcset="picture-330x207.jpg"><img src="picture-330x207.jpg"></picture> Since, the and tags are relatively new, we also include the tag for compatibility with old browsers. <picture> <source> <img> If you open the developer console, you will see different requests. 900 x 521 resolution is loaded on desktop 450 x 261 resolution is loaded on mobile devices For video, replacing element with the element will load the corresponding resolution of video. You will have to encode video in different resolution and store it on your system. At Genius, we use to automate all our video tasks. <picture> <video> FFmpeg Server-Side Resizing and Caching This works by sending the expected width/height of the image in the URL. The server resizes the target file on each request or serves it from the cache, if present. It is a relatively more complex solution because (i) the expected width/height needs to be calculated on the client to create the URLs for the attribute, and (ii) the parameters need to be understood by the server to, (iii) resize and cache the image. src Many CDN (Content Delivery Network) services like , , etc. come with image resizing capabilities. Even software optimized for static content delivery like it. For Node.js, we experimented with (based on ) and found it quite . Cloudflare Akamai Nginx support image-resizer Sharp fast WebP instead of JPG and PNG WebP leads to than JPG and PNG for the same quality. The only caveat being, it is . So care needs to be taken to serve the correct file (outlined excellently ). 20–35% smaller images supported only on Chrome, Opera and Chrome’s variants here WebP gives huge savings over JPG and PNG Our internal tools use to create WebP variants of all images. For instance, for we create an optimized JPEG, and a WebP, . We have a Nginx reverse proxy which detects support through the header of the HTTP request (looks for in the string) and serves the correct web image. Sharp picture.jpg picture.jpg picture.jpg.webp webp Accept webp JPEG, PNG and SVG optimizations We follow a simple 2 step rule to decide the image type and optimize them using . Sharp If it is a real-world photo or close to it, use JPEG. If there are limited number of colours and curves, use PNG/SVG (logos and icons) **Progressive JPEG**Besides being better for user experience, encoding JPEG as progressive also results in smaller image size. This mostly true only when the source image is over 10 KB. For less than 10 KB, prefer Baseline JPEG. Check out for a basic experiment. this article No GIFs. Use MP4. Graphics Interchange Format (GIF) was created in the stone age of internet and was not even meant for animation. The clearly states: GIF spec The Graphics Interchange Format is not intended as a platform for animation, even though it can be done in a limited way. , According to Google Delivering the same file as an MP4 video can often shave 80% or more off your file-size. Not only do GIFs often waste significant bandwidth, but they take longer to load, include fewer colours and generally offer sub-par user experiences. At Genius, we call from our tools to convert GIF to MP4. We have seen 700 KB GIF becoming 170 KB MP4. Here is a command to convert to (inspired by ). FFmpeg animated.gif video.mp4 this blog ffmpeg -i animated.gif -movflags faststart -pix_fmt yuv420p -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" video.mp4 Use HTTP/2 HTTP/2 is a new protocol for doing HTTP request. It is a binary, fully multiplexed protocol with support for header compression. In other words, it can use 1 TCP connection to load multiple files in parallel. That makes loading assets on HTTP/2 super fast, and it consumes less bandwidth because it compresses headers. This has a detailed performance analysis. css-tricks.com blog At Genius, we use AWS and as our endpoints. Both the services use HTTP/2 by default. Elastic Load Balancer CloudFront Clean unfinished entries in request queue We experienced an undesirable behaviour with our Single Page Application (SPAs) when the network becomes intermittent. The assets requested through and elements on previous screens remained in the network download queue even after the corresponding elements were removed from the HTML. This, in turn, delayed downloading of assets that needed to be displayed in the current view. <img> <video> Clearing the attribute on the element cancels the request instantly. Here is a React component in Typescript implementing this feature. src <img> For element, remove the attribute from element, and call on the mounted element. <video> src source load <video> With this, the browser cancels the download requests once the elements are removed from the HTML and sets as the status of these requests. (canceled) Canceled network requests Browser Caching Caching a resource correctly can help the browser avoid the network call altogether when the same resource is fetched again. There are two types of caches — one which persists the assets on the permanent storage (persistent) and ones which are memory based (non-persistent), hence application session specific. There are 4 types of cache in advanced browsers. They are encountered by a request in the following order. Memory Cache — as the name suggests, it caches assets in the memory. — a persistent cache used by service workers (uses ). Service Worker Cache Cache API — persists the response in accordance with the header set in the HTTP request. HTTP Cache cache-control HTTP/2 Cache — caches assets in memory which are pushed by HTTP/2. A fun article, “ ” by Yoav Weiss explains how all 4 work together. A Tale of Four Caches HTML Preload HTML is a declarative fetch, forcing the browser to request a resource before it is used. It can be used to preload assets into the HTTP and Memory Cache that will be needed in the immediate future. For instance, when someone is reading page 1 of a blog, we can preload images of page 2. preload <link rel="preload" as="image" href="sunlight-trees.jpg"> There is also declaration which works differently from . Here is a great , describing and in depth. prefetch preload article from Addy Osmani prefetch preload There is one caveat that we didn’t realize. Unlike which can load assets as , , , the declaration cannot load assets with a resource type. Simply put, the declaration in the block above will set the header as if the request is being made by the element. The doesn’t support attribute and will always set the header to . This makes undesirable because the browser will anyway re-fetch the resource when it is used in the HTML. preload image font script prefetch preload Accept <img> prefetch as="image" Accept */* prefetch HTTP Caching Browser largely has 3 modes, controlled by the HTTP header. HTTP cache [Cache-Control](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching/#cache-control) No caching — Do not store the response. Achieved by setting the response HTTP header . Cache-Control: no-store Re-validate every time — Store the response but validate with the server before using the response. Set the response HTTP header and the , uniquely identifying the delivered resource. If the doesn’t match with the latest file on the server, the server returns the new file else returns with HTTP status 304. Cache-Control: no-cache [ETag](https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching/#validating_cached_responses_with_etags) header ETag Don’t re-validate for a specified time — Store the response and don’t bother asking the server until a specified time is expired. Achieved by setting the . The seconds specify the time (max value of 31536000 (1 year)) until the client is going to serve the cached response instead of asking the server for new response. Cache-Control: public, max-age:<seconds> Use the 2nd mode when an asset corresponding to a URL may change in the future. For instance, if the image URL contains is just filename . [https://example.com/pic.jpg](https://example.com/pic.jpg) Use 3rd mode, when you can generate unique URLs for each resource. For instance, by having image content hash in the name . [https://example.com/pic-6a8sd6d82.jpg](https://example.com/pic.jpg) At Genius, all our image names contain hash. Hence, we always use the 3rd mode. Once we deliver the image, the client doesn’t ask the server unless the cache is deleted or a year has passed. This saves user as well as server bandwidth while delivering a seamless experience. Service Worker Caching are the new rage of web development for building rich offline experiences and much more. A service worker runs a script in the background, separate from the web page. The feature we are interested in is the ability to intercept the requests coming from the web app and provide the response from the cache that, as a developer, we manage for the app. Service Workers Asset caching strategy. Taken from the . offline cookbook Building a service worker can get quite complex. To make life easy, the good folks at Google have developed which only needs a configuration to install the service worker and manage the cache. For instance, to cache all the requests to , such that we don’t cache more than 20 images, for a maximum of 1 week, all we have to do is write a config like this. Workbox [https://example.com/image/](https://example.com/image/,) workboxSW.router.registerRoute(' workboxSW.strategies.cacheFirst({cacheName: 'images',cacheExpiration: {maxEntries: 50,maxAgeSeconds: 7 * 24 * 60 * 60,},cacheableResponse: {statuses: [0, 200]},})); https://example.com/image/(.*)', Check out this detailed . example The End Phew! That was a lot of information. Congratulations on reading until the end. If you find any aspect missing, please do mention in the comments below so that this document can be improved and made more useful. Thanks!

Amazon

Fetch

Google

Instantly

Mozilla

Progressive

Super

Target

Twitter

Too Long; Didn't Read

Get free API security automated scan in minutes

Building a Seamless Image and Video Experience for the Web

Building a Seamless Image and Video Experience for the Web

Too Long; Didn't Read

Companies Mentioned

@gaurav.lahoti

RELATED STORIES