Grafbaseis in the Application Development Tools Industry

Grafbase

Today is the first day of the WunderGraph launch week, it's our first launch week actually. We have a lot of things to share with you, like Open Federation, Cosmo, and more... What excites me the most is that today marks the beginning of a new era for . For a very long time, the GraphQL ecosystem has been dominated by a single company. However, this is about to change through two initiatives and the ecosystem that will emerge around them. GraphQL The first step is , an MIT-Licensed specification to build federated GraphQL APIs. The goal of Open Federation is to enable the GraphQL ecosystem to build federated GraphQL APIs without having to rely on Apollo's proprietary technology. It's a drop-in replacement for Apollo Federation, it's open source and free to use for everyone. Open Federation Most importantly, we're not announcing Open Federation alone. Open Federation is a joint effort by WunderGraph, Grafbase, Escape Tech, Neo4j, TailorTech, Sibipro, Soundtrack Your Brand, Travelpasses, and many more. The second step is , a specification for composed GraphQL APIs that goes way beyond what Apollo Federation offers, like support for composing not just GraphQL APIs, but also REST(OpenAPI), gRPC, AsyncAPI, and more. GraphQL Fusion . GraphQL Fusion will live under the GraphQL Foundation The goal of Fusion is to enable the composition of GraphQL APIs with a diverse set of vendors and technologies. Fusion is a joint effort by ChilliCream, The Guild, Hasura, IBM, Solo.io, AWS AppSync, and WunderGraph. Instead of every vendor building their own proprietary composition technology, we're building a single open-source specification that everyone can implement, so that we can all work together to build a better GraphQL ecosystem. For today, we're focusing on Open Federation, but stay tuned for more information about GraphQL Fusion. You'll definitely hear more about it during the GraphQL Conference by the GraphQL Foundation. We'll be there as well, so make sure to say hi at our booth. The History of Open Federation I'm the creator of graphql-go-tools, a GraphQL library for Go that is used by many companies to build GraphQL Gateways and other GraphQL tools. I've been working on this library for more than 5 years now and it has been a great success. Almost 3 years ago, I started adding support for . As excited as I was about the idea of Federation, the community was not ready for it yet. I added support for Subscriptions years ago, but demand for it was very low, so my focus shifted to solving other problems. Apollo Federation to graphql-go-tools Fast forward to 2023, and suddenly things have changed fundamentally. The GraphQL ecosystem has grown a lot and the demand for Federation has increased significantly. At the same time, Apollo started closing down their open-source ecosystem with license changes, price increases, and more. All of this led to a lot of people reaching out to us and asking for help. Some of them, like TailorTech, were even willing to sponsor the development of a Federation v2 compatible Gateway and composition tooling for graphql-go-tools. Big enterprises helped us a lot with testing or providing GraphQL schemas of their federated Graphs to test our implementation. But there was a problem we had to overcome first. What actually is Apollo Federation? This might sound like a silly question, but it's actually not. At this point, we should all have some high-level understanding of what the Federation is. But have you ever looked at the details, like the specifications? Here's the of Apollo Federation v2. It's nine Directives and one Scalar, that's it. No comments, no explanations, no examples, nothing. It's everything but a specification and it actually leaves all the important questions unanswered. "Specification" For example, how do you actually implement a Federation Gateway? What is the Gateway supposed to do? What are the rules to validate a federated GraphQL schema? How do you actually compose federated GraphQL APIs? What are the rules to check if two or more subgraphs are compatible? In comparison, the GraphQL Specification describes in a lot of detail how a GraphQL Server should behave. It contains numerous examples for validation and execution rules, as well as descriptions of basic algorithms like how to merge selection sets or how to execute a GraphQL Query. If you want to understand Apollo Federation in detail, you have to read a lot of documentation in different places and even then, you'll still have a lot of questions. You might be able to reverse-engineer the behavior of the Apollo Gateway and Router, but even if you do, there are still some behaviors that seem random and are not documented anywhere. So we asked ourselves, how can we build tools to compose subgraphs and a Router / Gateway to resolve federated GraphQL Queries without having a proper description of "our understanding" of Federation? The birth of Open Federation That's when we decided to start writing down our own understanding of the Federation. By writing down all our knowledge, you're able to criticize it and help us improve it. If you find gaps or mistakes, we can collaborate to find a common understanding of what Federation is, how composition works, and how a Gateway should behave. From that point on, multiple vendors can implement the same specification so you're free to choose the best solution for your use case. You're not locked into a single vendor anymore, you can switch to a different vendor at any time. You can even if you want to. In addition, you can suggest improvements to the specification and actually have an impact on the future of federated GraphQL APIs. build your own Gateway The goals of Open Federation We don't believe that there's a future for proprietary API Composition solutions. It's very unlikely that a single vendor with a closed-source solution is capable of serving the market with the best possible solution. We've seen the opposite in the past with OpenAPI. Before OpenAPI, there were many proprietary custom solutions to describe and manage . However, OpenAPI has become the de-facto standard for REST APIs and is supported by many vendors and tools. It's a great success story and has led to a lot of innovation in the REST API space. REST APIs We believe that Open Federation can have a similar impact on the GraphQL ecosystem. An open specification will make federated GraphQL much more accessible and attractive to a lot of companies and therefore will lead to a bigger adoption of GraphQL in general. This in turn will attract more vendors to the GraphQL ecosystem, which will lead to more innovation and healthy competition. After all, it's in the best interest of the GraphQL ecosystem to have a diverse set of vendors and technologies. By standardizing the composition of GraphQL APIs, vendors can focus on their strengths and collaborate with complementing solutions. The key is Composable Architecture and interoperability. Different implementations of the specification can focus on different use cases and technologies. One vendor might focus on a Gateway that is optimized for performance by using Rust or Go, while another vendor might optimize for extensibility, e.g. by writing the Gateway in NodeJS. At this point, you might be wondering, what's the difference between Open Federation and Apollo Federation? What's the difference between Open Federation and Apollo Federation? Apollo Federation is not a complete specification. It's a set of directives that, if used correctly on a subgraph, will make it compatible with the Apollo Gateway and Router. Open Federation builds on top of the same directives to specify subgraphs. It's a specification that describes in detail the rules to compose subgraphs, and how federated GraphQL requests should be planned and executed. Open Federation can be seen as a drop-in replacement for Apollo Federation, just that it's way more specific, open source, and free to use for everyone. What's the relationship between Open Federation and GraphQL Fusion? For a lot of use cases, federated GraphQL APIs will be more than enough, and there's no reason not to adopt or use Open Federation. However, there will be use cases where you want to compose not just federated GraphQL APIs, but also REST APIs, gRPC APIs, AsyncAPI, and more. This is where GraphQL Fusion comes into play. Conclusion Federated GraphQL is a mature technology and has been used in production for years. GraphQL Fusion is still in its early stages and will take some time to get to the same level of maturity. We bet on both technologies and believe that they will complement each other very well. Time will tell if both technologies will live alongside each other or if one will eventually dominate the other. What matters most to us is that we're standardizing the composition of APIs to build a better GraphQL ecosystem. Also published . here If you haven't already, make sure to check out and have a look at the specifications. We're looking forward to your feedback and contributions. Open Federation

Open Federation - An MIT-Licensed Specification to Build Federated GraphQL APIs

I have this unexplained urge to write. I sometimes feel it’s the only way I can express myself. But this has been a recent development, 2 years in the making. What gave it depth, was my unknown, self-drive to find out more about Blockchain Technology. I struggle to pass an article and not read it, I must say a lot faster these days and sometimes skimming to get the gist to assess where I can use or store it. Added benefits, of course, has been an improved vocabulary and to my wife’s happiness improved spelling 😊 Not something I planned and certainly not something friends and family expected from me. Building up the knowledge base gave me the power to write about a subject I am passionate about — Decentralization. Blockchain is the vehicle that will drive this concept and as I continue learning about the subject, find it has a resounding effect on a borderless network of possibilities forward. Before you think this article is about me, my point here is, don’t wait for something to come to you, don’t think someone owes you something, don’t think about the money you should be earning for all the hours you put in. Rather think about the contribution you want to make, like a friend said, “think about what you can bring to the table”. I can carry on with cliché’s and I’m not a ‘feel good quote’ kind’a guy, but it really is about how I think business ethics will change and how up and coming generations should embrace this opportunity, this gap they are being given to change the status quo. “This way the money will follow you” Women suddenly have this upper power… yes, the young generations coming through the ranks have an incredible independency value to them and this challenges all the perimeters. The knowledge that women need today to be equal to ‘men (if you know what I mean) is equally accessible and the job requirements are no longer fixed, (rather, remote…, and will become more so as automation evolves). This series “Women in Blockchain” is to uncover just that, open up seemingly difficult to reach career area’s and explaining a new industry that will see a career path into a very bright future. My first interview then is with Jillian Godsil Jillian Godsil: Conference Chair, Broadcaster, Journalist, Author, Keynote, Author, Homeless Advocate, ran the EP14 Race, Law Changer Q&A Give us a summary of your background and how you got into Blockchain & Digital Currency / Cryptocurrency? “For the first time in my working career, my age and gender is an advantage. I’m a woman and I’m in my fifties (early :) in charge of my destiny for others.” Hi, I’m Jillian Godsil co-founder of Blockleaders.io and Editor in Chief. I have managed some great achievements with EOS in Dublin, named in the top 50 Women in Blockchain Technology and in general busy with my passion, the media. I am a freelance journalist for the Irish Times, the Irish Independent and many more broadsheets and business publications. I am an advisor for a number of companies working in the Blockchain industry and travel the world speaking and chairing Blockchain conferences. I host two weekly terrestrial radio shows, DubinCityFM and East CoastFM, and was nominated for a national radio award, an IMRO, in 2019. It wasn't always like this, a few years ago I lost everything and had to start from nothing with my two girls. I had to crawl my way back into society, but with a redefined purpose. I remember fighting for my rights in the constitutional court and winning! By doing so, I ran in the European Parliamentary elections in 2014 and while she did not win a seat I earned 11,500 votes. This is where blockchain started for me. By September 2017 she was travelling the world, speaking and chairing at blockchain conferences, not just for me, but for humanity. 2) Tell us about your current position, responsibilities and some day to day activities that you look forward to? I am co-founder and Editor in Chief of Blockeladers.io — and have been awarded the 2019 AI and Blockchain Journalist in London last month. Here I do video and written articles/profiles on all things blockchain-related. I do a weekly podcast for EOS Doblin. I work as a consultant and PR adviser to a number of projects in this space. I chair, moderate and keynote at blockchain events around the world — love this side to my career I have two terrestrial radio shows here in Ireland that has nothing to do with blockchain — although I do manage to sneak a lot of blockchain data into them lol — DublinCityFm and EastCoast FM. And this year I was nominated for a national radio award from IMRO. 3) If you had to be critical about yourself, which part of your career would you like to improve on? Well, my kids might say the financial side — it’s taken me 12 years to turn the financial corner but finally, finally it is happening as I start to earn money again. Hurrah! 4) What advice do you have for children/youth (10–18 years ) today especially young women about the Blockchain world? How do we nurture interest in this new Technology? It doesn’t matter what you do — blockchain is like the internet — it is going to be pervasive in all our lives but to young people I point to the thinking in the blockchain which is exciting in this very broken world of ours. The thinking is disruptive, the thinking is innovative and the thinking is based on making a better world. So it doesn’t matter what a young person or woman wants to do — blockchain will be in their future. My only exhortation is for young people to own this — like they now do with climate change — and make sure the old fools don’t screw it or the world up. 5) What kind of skill set would you be looking at in job applicants? Same as any other industry — enthusiasm, intelligence, creativity, problem-solving skills — all laced with kindness and a concern to make the world a better place. 6) Blockchain to me is a forward-thinking force of snowballing Technology, but where I’m from a lot of people still see Bitcoin as a scam. How do we bridge the gap between the Technology and people that are skeptical about it? We just keep showing up and talking. That for me is what I am put on this earth to do — talk and communicate endlessly about blockchain. And have fun too — much easier to persuade people to look at new opportunities if the people involved are enjoying the new world. 7) Can you please provide the details of an educational facility that is in your area or country if anyone reading this article desires to find out more about Blockchain. Well it’s actually over in the UK but two companies to watch are Cointelligence which runs an Education Academy and CoinBurp that are doing education videos featuring Calum Best (George Best’s son and influencer) In terms of reading — please visit my website http://blockleaders.io/ Or EOS podcast — https://eosdublin.podbean.com/ Or my new ICO SOFA channel https://www.icosofa.com/channel/jillian-godsil/?view=about Or my DublinCityFM radio show https://www.dublincityfm.ie/shows/making-a-difference/ 8) What is your current opinion on the Digital Assets, ( Cryptocurrency ) market and share with us your opinion on where you think this technology will take us? Tokenise everything! Digital Assets have existed for a long time but now they are moving into blockchain technology which is very exciting. It’s the future from property, DNA, music etc etc 9) What new exciting projects are you currently working on or planning? I am working on a very serious project called Europechain which offers GDPR compliant blockchain — really cool project. 10) What is your favorite Blockchain project and why? CoinBurp — they are great guys but they have recruited Calum Best as an ambassador. I have to admit to having a girl crush on Calum. So that makes CoinBurp my favorite project. What an amazing story and again it shows to me what humans are actually capable of when they are confronted with difficult circumstances or when they follow their passions. Either way, for me #havefunwithit is my ‘take-home’ and you know it becomes easy to talk about something we enjoy, that energy is contagious and in the Blockchain community at large a contagious Decentralized network is spreading because of woman like you Jillian. My next guest is Maria Grazia Maria Grazia Vigliotti, Ph.D: Cybersecurity Expert, Blockchain, Audit of Smart Contracts, Author “The Executive Guide to Blockchain” Q&A Give us a summary of your background and how you got into Blockchain & Digital Currency / Cryptocurrency? Thank you for this question. I have studied Computer Science and I have Ph.D. in Computing from one of the most prestigious Computing Departments in the world: Imperial College London. I carried out exciting research at Imperial on both cybersecurity and cryptography, and it was a small step to learn about blockchain in general. In short, I could see how with my set of skills I could positively influence the technology. 2) Tell us about your current position, responsibilities and some day to day activities that you look forward to? There are not many women in technology, so in 2017 I decided I wanted to make a mark and I set up my own company, Sandblocks Consulting. At the same time, I started to work on an extraordinary start-up Gradbase, that inserts academic qualifications on the blockchain. Both companies have grown enormously, and that has given me the time to write a book on Blockchain “The Executive Guide to blockchain”, which will be published by Palgrave in February 2020 -see The goal of the book is to explain blockchain technology to everyone who does not have a technology background, but an interest in understanding the principles and the applications of blockchain. https://www.amazon.co.uk/Executive-Guide-Blockchain-Contracts-Currencies/dp/3030211061 Going back to my typical day at work, because I run businesses, I do not have a typical day. Every day is different, and I personally love it. Some days I meet customers to understand what they need, or I lead my technical team to ensure deliveries of projects. I also participate in meetings, conferences to deliver thought leadership on blockchain. I run training sessions for executives and technical teams. I always need to make time to plan future activities to keep the companies afloat and understand where the market is going to remain relevant. ? 3) If you had to be critical about yourself, which part of your career would you like to improve on When I started my career, I wish I had more confidence and I had been more assertive about advancing my career. Women tend to be overcritical about themselves and do not put themselves forward. We are making disfavor to ourselves and to other women: there are fewer role models and mentors that will help women to advance. Advancing the career does require to ask for promotion rather than waiting to be recognized. One needs to feel confident that promotion is deserved. Women mentors can help other women to increase the confidence and to find ways to ask for promotion. 4) What advice do you have for children/youth (10–18 years ) today especially young women about the Blockchain world? How do we nurture interest in this new Technology? My best advice is do not be afraid to try something new, even if it feels alien. Technology will become part of your life one way or another, learn to understand it now and you will be in a great position to make it work for you at a later stage. I also would encourage young people to have a look a free resources where they can learn technology in a way that suits them. Have a look at some amazing free resources: — This website is full of interesting courses that if I had the time, I would take myself. It has been done with the help of great engineers, and it only requires an internet connection! https://studio.code.org/courses — This is a fantastic project where young people can learn to build their own computers. It requires to purchase some hardware. https://www.raspberrypi.org — Another great online resource for young people to learn about coding from MIT. http://appinventor.mit.edu It does not matter where they start: what matters is, that they start now! 5) What kind of skill set would you be looking at in job applicants? Beyond the specific skills of the job, the most important skills we are interested in are good communication skills and independence. Being able to clearly communicate with others eliminates friction and makes work more enjoyable. I do not micromanage people, I expect people to be able to sort out problems or to come back and tell me what they need to sort out the problems in their job. That is independence for you. In my view, these skills must be part of every employee to become a successful company. 6) Blockchain to me is a forward-thinking force of snowballing Technology, but where I’m from a lot of people still see Bitcoin as a scam. How do we bridge the gap between the Technology and people that are skeptical about it. I think the best way to gain a balanced view of the technology is to learn more about it and deploy it in some way. Let’s consider a similar problem in a different situation. If you only have used a horse to travel, any modern city could look scary and dangerous. But once one learns how to drive a car, and what are the rules for driving on the road, then the perspective completely changes. Similarly, learning about cryptocurrency and blockchain will help people to understand how technology can be used, and skepticism will disappear. 7) Can you please provide the details of an educational facility that is in your area or country if anyone reading this article decides to find out more about Blockchain? I teach about blockchain at TechUK ( ) in London, which is an organization with 850 companies from around the UK. I offer a one-day overview about the technology and its deployment. There are several online courses that people can take, and some of them are also free, for example, .so a great place to start. https://www.techuk.org https://www.edx.org/professional-certificate/uc-berkeleyx-blockchain-fundamentals 8) What is your current opinion on the Digital Assets, ( Cryptocurrency ) market and share with us your opinion on where you think this technology will take us? The cryptocurrency market is overcrowded at the moment, according to Coinmarketcap ( ) there are over 4904 cryptocurrencies at the moment. I believe that similarly to the Dot-com bubble, in the next five years, we will see what projects will survive and these will shape the future market. https://coinmarketcap.com 9) What new exciting projects are you currently working on or planning? Currently, I am working on ensuring more women take part in blockchain projects. I am working on creating training for those women. 10) What is your favourite Blockchain project and why? I have two favorite projects: are Algorand and Elixxir . These are two projects that deploy blockchain in a way that is useful to society, and they are very robust from a technical point of view. https://www.algorand.com https://elixxir.io Thank you, Maria, what a wealth of information. I’ll be sure to look into the courses and sharing more information on them and I like your analogy of the horse and car, (I think of it as a wheel, it still stays round, but we improve the rim and tires or change them to suit the road conditions), educating yourself beyond what is expected of you, will define your pathway into the technological future. I still think there will be some great power struggles between up and coming students, women and the current upper-level C-Suites, unless they take to book and adapt, (fast). “Women in Blockchain” articles from Morne Olivier LinkedIn Writer | Marketer | Blockchain Consultant Morne Olivier #Disclaimer LinkedIn Medium Hackernoon Twitter Telegram Telegram for Passive Income Uptrennd Steemit FaceBook Designs in Canva

Women in Blockchain | Changing the Status Quo

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de;
(2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl;
(3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de;
(4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. Table of Links Abstract and Introduction
Related Work
Methodology
Experimental Design
Experimental Results
Conclusion
Ethics Statement and References II. RELATED WORK Exploiting Item Features: Content-based recommender systems incorporate item tags, item-item similarity, and userside features. Item-item similarity typically maps the tag clouds of items into a latent space and computes distances between the embedding points. This can be combined with interaction-based methods that employ latent-space techniques including deep learning (e.g., [2][8], [12]–[14]) or graphbased inference (e.g., [15], [16]). These methods excel in performance, but experimental results often benefit from a large fraction of favorable test cases. For example, when the model is trained with books by some author, predicting that the user also likes other books by the same author is likely a (near-trivial) hit. In our experiments, we ensure that such cases are excluded. Exploiting User Reviews: The most important user features are reviews of items, posted with binary likes or numeric ratings. Early works either mine sentiments on item aspects or map all textual information to latent features using topic models like LDA or static word embeddings like word2vec (see, e.g., [17]). Recent works have shifted to deep neural networks with attention mechanisms [9]–[11], [18]–[20], or feed review text into latent-factor models [21]–[23]. Some works augment collaborative filtering (CF) models with user text, to mitigate data sparseness (e.g., [24]–[26]). However, pre-dating the advent of large language models, all these methods rely on static word level encodings such as word2vec, and are inherently limited. As a salient representative, we include DeepCoNN [18] in the baselines of our experiments. Transformer-based inference: Recent works leverage pretrained language models (LMs), mostly BERT, for recommender systems in different ways: i) encoding item-user CF signals with transformer-based embeddings, ii) making inference from rich representations of the input review texts, or iii) implicitly incorporating the “world knowledge” that LMs have in latent form. An early representative of the first line is BERT4Rec [27], [28], which uses BERT to learn item representations for sequential predictions based on item titles and user-item interaction histories, but does not incorporate any text. The P5 method of [29] employs a suite of prompt templates for the T5 language model, in a multi-task learning framework covering direct as well as sequential recommendations along with generating textual explanations. We include a text-enriched variant of the P5 method in our experiments. The advances on large language models inspired approaches that leverage LLM “world knowledge”. Early works use smaller models like BERT, to elicit knowledge about movie, music and book genres [30]. Recent studies are based on prompting large autoregressive models, such as GPT or PaLM, to generate item rankings for user-specific recommendations [31], [32] or to predict user ratings [33], in a zero-shot or few-shot fashion, using in-context inference solely based on a user’s item titles and genres. Closest to our approach are the methods of [34], [35], using BERT to create representations for user and item text, aggregated by averaging [34] or k-means clustering [35]. The resulting latent vectors are used for predicting item scores. A major limitation is that the text encodings are for individual sentences only, which tends to lose signals from user reviews where cues span multiple sentences. Also, BERT itself is fixed, and the latent vectors for users and items are pre-computed without awareness of the prediction task. Our experiments include the method of [34], called BENEFICT, as a baseline. This paper is available on arxiv under CC 4.0 license. Authors: (1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de; (2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl; (3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de; (4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. This paper is available on arxiv under CC 4.0 license. Authors: (1) Ghazaleh H. Torbati, Max Planck Institute for Informatics Saarbrucken, Germany & ghazaleh@mpi-inf.mpg.de; (2) Andrew Yates, University of Amsterdam Amsterdam, Netherlands & a.c.yates@uva.nl; (3) Anna Tigunova, Max Planck Institute for Informatics Saarbrucken, Germany & tigunova@mpi-inf.mpg.de; (4) Gerhard Weikum, Max Planck Institute for Informatics Saarbrucken, Germany & weikum@mpi-inf.mpg.de. Table of Links Abstract and Introduction Related Work Methodology Experimental Design Experimental Results Conclusion Ethics Statement and References Abstract and Introduction Abstract and Introduction Related Work Related Work Methodology Methodology Experimental Design Experimental Design Experimental Results Experimental Results Conclusion Conclusion Ethics Statement and References Ethics Statement and References II. RELATED WORK Exploiting Item Features: Content-based recommender systems incorporate item tags, item-item similarity, and userside features. Item-item similarity typically maps the tag clouds of items into a latent space and computes distances between the embedding points. This can be combined with interaction-based methods that employ latent-space techniques including deep learning (e.g., [2][8], [12]–[14]) or graphbased inference (e.g., [15], [16]). These methods excel in performance, but experimental results often benefit from a large fraction of favorable test cases. For example, when the model is trained with books by some author, predicting that the user also likes other books by the same author is likely a (near-trivial) hit. In our experiments, we ensure that such cases are excluded. Exploiting Item Features: Exploiting User Reviews: The most important user features are reviews of items, posted with binary likes or numeric ratings. Early works either mine sentiments on item aspects or map all textual information to latent features using topic models like LDA or static word embeddings like word2vec (see, e.g., [17]). Exploiting User Reviews: Recent works have shifted to deep neural networks with attention mechanisms [9]–[11], [18]–[20], or feed review text into latent-factor models [21]–[23]. Some works augment collaborative filtering (CF) models with user text, to mitigate data sparseness (e.g., [24]–[26]). However, pre-dating the advent of large language models, all these methods rely on static word level encodings such as word2vec, and are inherently limited. As a salient representative, we include DeepCoNN [18] in the baselines of our experiments. Transformer-based inference: Recent works leverage pretrained language models (LMs), mostly BERT, for recommender systems in different ways: i) encoding item-user CF signals with transformer-based embeddings, ii) making inference from rich representations of the input review texts, or iii) implicitly incorporating the “world knowledge” that LMs have in latent form. Transformer-based inference: An early representative of the first line is BERT4Rec [27], [28], which uses BERT to learn item representations for sequential predictions based on item titles and user-item interaction histories, but does not incorporate any text. The P5 method of [29] employs a suite of prompt templates for the T5 language model, in a multi-task learning framework covering direct as well as sequential recommendations along with generating textual explanations. We include a text-enriched variant of the P5 method in our experiments. The advances on large language models inspired approaches that leverage LLM “world knowledge”. Early works use smaller models like BERT, to elicit knowledge about movie, music and book genres [30]. Recent studies are based on prompting large autoregressive models, such as GPT or PaLM, to generate item rankings for user-specific recommendations [31], [32] or to predict user ratings [33], in a zero-shot or few-shot fashion, using in-context inference solely based on a user’s item titles and genres. Closest to our approach are the methods of [34], [35], using BERT to create representations for user and item text, aggregated by averaging [34] or k-means clustering [35]. The resulting latent vectors are used for predicting item scores. A major limitation is that the text encodings are for individual sentences only, which tends to lose signals from user reviews where cues span multiple sentences. Also, BERT itself is fixed, and the latent vectors for users and items are pre-computed without awareness of the prediction task. Our experiments include the method of [34], called BENEFICT, as a baseline.

Recommendations by Concise User Profiles from Review Text: Related Work

UNIX/Linux system administrators the world over regularly use log files to get to the bottom of outages and malfunctions. An indispensable tool in that regard is , particularly its flag ( ). When we're in a Kubernetes world, we'd love to use something similar. tail(1) follow mode -f We're in luck. The command has two flags that are most helpful for watching live log streams: the aptly-named flag and the context-limiting . logs --tail -n kubectl logs -pod- my name --tail -n10 This one-two combo is super helpful for tracing ongoing issues on a running system. Limiting the context with prevents us from fixating on old problems that may have since been resolved. Streaming the log as it happens gives us the opportunity to poke and prod at the system, seeing new log entries appear as we do things in the application or service under scrutiny. -n While tailing, you can hit <Enter> a few times to insert lots of blank lines, thus demarcating the stretches of time between trying things in the front-end. Pro Tip: When Pods Crash $ kubectl logs crashing-pod
[Mon Feb : : UTC ] starting up...
[Mon Feb : : UTC ] startup complete; initializing.
[Mon Feb : : UTC ] - frobbing the data cache...
[Mon Feb : : UTC ] - allocating grokbase fooblers...
[Mon Feb : : UTC ] - reticulating splines...
[Mon Feb : : UTC ] - reclaiming unused memory...
[Mon Feb : : UTC ] - checking on friends family...
[Mon Feb : : UTC ] initialization complete; entering main loop. 10 16 29 19 2020 10 16 29 34 2020 10 16 29 34 2020 10 16 29 34 2020 10 16 29 34 2020 10 16 29 34 2020 10 16 29 34 2020 in and 10 16 29 34 2020 What I really need is the end of the log buffer from the failed / crash instance: $ kubectl logs -p crashing-pod
[M Feb : : UTC ] starting up...
[M Feb : : UTC ] startup complete; initializing.
[M Feb : : UTC ] - frobbing the data cache...
[M Feb : : UTC ] - allocating grokbase fooblers...
[M Feb : : UTC ] - reticulating splines...
[M Feb : : UTC ] - reclaiming unused memory...
[M Feb : : UTC ] - checking friends and family...
[M Feb : : UTC ] initialization complete; entering main loop.
[M Feb : : UTC ] ...
[M Feb : : UTC ] ...
[M Feb : : UTC ] ...
[M Feb : : UTC ] ...
[M Feb : : UTC ] UNMITIGATED DISASTER!
[M Feb : : UTC ] THREAD PANIC detected; no assurances given.
[M Feb : : UTC ] IT'S A BAD SCENE, MAN!
[M Feb : : UTC ] on 10 7 18 20 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 in on on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 on 10 7 18 35 2020 <crashed> With information in hand, I can continue my diagnostic efforts and hopefully resolve this early morning nuisance. that Previously published at https://starkandwayne.com/blog/silly-kubectl-trick-4-getting-at-those-logs/

Kubernetes Explained Simply: Getting At Those Logs [Part 4]

For any API-first company, implementing the right API analytics platform is critical for tracking the utilization of your APIs and to discover any performance or functional issues that may be impacting customers. There are a variety of open-source projects you can leverage to build a complete API analytics platform. Before jumping into building an API analytics solution yourself, you should first list out your requirements and use cases. Not all tools support all use cases directly, and may require heavy investment in development and integration. API analytics build requirements Need to answer engineering or business questions Engineers commonly turn to API logs and metrics to understand what’s happening over their APIs, inspect payloads, and root cause issues with their services that come up. capabilities is a must for engineers that want to depend on their API analytics solution to put out the (hopefully far and few between) fires due to an API outage or reliability issue. Because real-time logging can raise the compute and storage cost, not all analytics infrastructure maintains a real-time pipeline. On the flip side, only the most recent data is needed to answer hair on fire engineering problems, so data can be after a short time period such as after 24 hours. Real-time API logging retired Engineering and product leaders make strategic decisions based on lines, not dots. Business questions are answered from historical trends in data, which may be over months or even years. This means your API analytics build should be capable of storing data for long retention periods such as for multiple years. This also requires infrastructure that can roll up and compress your data as storing and running aggregations on raw event logs will cripple your analytics infrastructure Real-time alerting and monitoring? While building dashboards and throwing them up on a monitor can be a great way to monitor your metrics, it still requires manually checking them periodically. To become more proactive, many API analytics builds also have some sort of monitoring and alerting features. These can vary from simple threshold-based alerts that sends an email when a metric reaches a certain value, to complex monitoring rules and workflows that can perform complex aggregations in real-time and route alerts to incident response platforms like PagerDuty and BigPanda. Some customer-facing teams like developer relations and customer success may want to build automated workflows to notify customers or internal specific internal stakeholders based on complex user behavioral flows. Access data anywhere vs controlled access The post-COVID world has accelerated the shift towards work from home. This means internal users of your API analytics build may need to access metrics on home networks or on the go without a VPN. Placing unnecessary security restrictions to access the API data may limit the value your API analytics provides to your company or can even backfire creating bad habits such as password sharing or exporting large amounts of data to a personal device rather than using the API analytics build the way it was designed. On the flip side, providing self-service API analytics from anywhere also means having strong security and access control. If you think typical enterprise access control like single sign on and role-based access control may be needed in the future, you should plan your API analytics build accordingly even if not needed immediately. Ripping out and changing authentication and authorization design is not an easy task and could force a complete rewrite. You don’t want to be caught off guard and be the engineer who failed at foreshadowing typical future enhancements. Flexibility of visualizations While most analytics platforms can display event data or plot basic single-value metrics over time, you platform may also be used for more advanced and specialized analysis like or cohort retention analysis. These are common queries used by business functions like marketing and growth teams but rarely used by developers themselves. Yet, building a funnel can be quote challenging if the right data model was not used. funnel analysis Many times, you don’t know the types of queries that need to be displayed but should choose a project that enables flexibility in both the data model and the visualization layer. Comparison There are a variety of open-source analytics and monitoring projects out there. Some are focused on monitoring infrastructure metrics like Kibana and Grafana. While others are more focused on web analytics like Matomo (also known as Piwik). While none of these are designed for API products, you could develop custom code to piece a few components together to build an open-source API analytics platform Kibana Kibana is one of the de facto open-source log visualization tools out there for engineers. It’s part of the official ELK stack (Elasticsearch Logstash Kibana) which makes it one of the quickest visualization tools to get up and running due to it’s tight integration with Elasticsearch relative to tools like Grafana which are far more complex to set up. Elasticsearch itself is well suited for high-cardinality, high-dimension log data which is a must for API logs. The downside is that Kibana is compatible with Elasticsearch. If you want to visualize data stored in a SQL database or other data store, you’ll need to look elsewhere. only While quick to set up, Kibana is also quite limited in the types of visualizations and flexibility supported. Kibana’s primary use case is to provide log search and light analysis on raw event data rather than offering a true API monitoring tool. For debugging use cases, this may be sufficient, but popular business metrics like funnels and retention analysis cannot be performed by Kibana limiting it’s application outside of engineering teams. Kibana and Elasticsearch is designed to be paired with a Logstash instance which enables you to design a logging pipeline to process and enrich API logs such as normalized any HTTP headers or add geo IP information to each API call. Keep in mind Logstash cannot perform aggregations across multiple events at a time. Such processing requires a separate cluster-computing framework for map-reduce operations like Spark or Hadoop. Kibana is designed for use cases where you want to explore your data in an ad hoc fashion rather than create a daily dashboard. You’re able to leverage Elasticsearch query DSL or Lucene query syntax providing great flexibility, but these do have a steep learning curve. By default, Kibana is purely a visualization tool which means things like alerting, anomaly detection, and authentication are separate. This means anyone with access to your Kibana endpoint can access your data so you shouldn’t have it public. However, you can purchase and install the Elasticsearch X-Pack to gain some monitoring functionality along with access control. Grafana Unlike Kibana which focuses on log search, Grafana focuses on time-series based metrics. You can visualize data in a variety of databases including Elasticsearch, InfluxDb, OpenTSDB, Graphite, and Prometheus. Grafana does one thing, and one thing really well, which is visualizing time series metrics stored in a database with beautiful dashboards. This does leave everything else up to you including configuring your data source and processing your data into a time series metric that can be displayed by Grafana. Compared to Kibana, Grafana only works on time-series data already stored in a database and does not have any real-time log search nor a way to browse or explore your raw data in an ad hoc way. The primary use case for Grafana is to design a dashboard to monitor time-series metrics regularly such as on a TV in the office. For example, you may want to display disk utilization, system CPI, and requests per minute for your servers. Grafana has a lot of options to display your metrics in the way you want, like showing storage capacity using base 2 units and percentile based metrics. Due to it’s time-series based architecture, Grafana’s application for high-cardinality, high-dimension analytics on API calls is limited. Instead, you need to spend time deciding exactly what specific time-series metrics you want to track ahead of time and model your data accordingly. This also limits Grafna’s use case for the self-service data exploration that business users may be looking for where you’re looking to segment with multiple or correlate multiple dimensions in your data. group by’s Compared to Kibana, Grafana is known for built-in support for authentication and access control. You can also hook up your Grafana instance to an external Lightweight Directory Access Protocol (LDAP) server or SQL server to better control access in an enterprise setting. You can also connect Grafana to an incident response platform like PagerDuty to create and trigger alerts from your Grafana instance. Keep in mind these alerts are limited to the same time series metrics you are already monitoring in Grafana. Grafana does have a separate product called Loiki, which provides some of the log exploration features that Kibana has. Jaeger Like Grafana which specializes in time-series metrics, Jaeger does one thing and one thing well, which is visualizing distributed traces. This makes Jaeger quite a bit of a different tool than Grafana and Kibana in that each trace is created and viewed in isolation vs monitoring metrics or logs over time. A trace is a snapshot of all context and timing info as a request propagates through a service mesh or hits various services in a microservice architecture. Because trace generation is expensive, sampling is usually employed to snapshot every X requests or specific criteria. Unlike Grafana which focuses on monitoring time-series metrics and Kibana which focuses on log search, Jaeger focuses on root causing specific issues with a service mesh or dependency issues. Jaeger supports multiple data sources like Grafana including Cassandra and Elasticsearch. Since traces are created in isolation, the only view is a trace view as expected. There is no way to create trends over time. Jaeger also doesn’t have any alerting or monitoring features so you’ll want to still have a Grafana or similar instance handy. Moesif Many API teams find needing a multitude of tools for their monitoring needs. Grafana has time-series metrics whereas Kibana enables log search. A faster option is an end to end solution designed specifically for API products like . Unlike Grafana where time-series metrics need to be preplanned, Moesif is designed for ad hoc data exploration using high-cardinality, high-dimension API analytics which is a must for product and engineering leaders looking to make strategic decisions from their API usage data. Moesif Compared to infrastructure monitoring tools like Kibana and Grafana, Moesif leverages a which enables you to align your API metrics to business goals by tying API metrics to individual customers rather than tied to infrastructure. This is also known as which enables understanding complex user flows holistically across multiple user actions and API calls rather than looking at each time-series in isolation. user-centric data model user behavioral analytics A classic example of user behavioral analytics is monitoring a and then breaking it down by user acquisition channel to decide where to invest marketing dollars or viewing for different features or endpoints to make your API products more sticky. This also enables tracking higher level account health for security research and customer success. conversion funnel user retention Compared to Grafana and Kibana, Moesif also provides elaborate real-time alerting and reporting features. This enables you to see which endpoints are causing the most performance issues broken down by each customer email. With behavioral emails and workflows, you’re able to scale customer outreach and support efforts with automated emails with a set of steps.

Grafbase

grafbase.dev

-- employees

Since n.d.

COMPANY RANKING

GRAFBASE

EVERGREEN INDEX #4623

Grafbase WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

Grafbase

grafbase.dev

-- employees

Since n.d.

COMPANY RANKING

GRAFBASE

EVERGREEN INDEX #4623

RELATED COMPANIESRANK

STORIES ABOUT GRAFBASE

Grafbase WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!