DotCalis in the Enterprise Collaboration Industry

DotCal

dotcal

Creating Staggered Animation in Flutter: A Micro-Interactions Guide

ফ্লটারে স্তব্ধ অ্যানিমেশন তৈরি করা: একটি মাইক্রো-ইন্টার্যাকশন গাইড

Erstellen von gestaffelten Animationen in Flutter: Ein Leitfaden für Mikrointeraktionen

Creación de animaciones escalonadas en Flutter: una guía de microinteracciones

Création d'une animation échelonnée dans Flutter : un guide des micro-interactions

फ़्लटर में स्टैगर्ड एनीमेशन बनाना: एक माइक्रो-इंटरैक्शन गाइड

Flutter でスタッガードアニメーションを作成する: マイクロインタラクションガイド

Flutter에서 시차 애니메이션 만들기: 마이크로 상호작용 가이드

Criando animação escalonada no Flutter: um guia de microinterações

Создание шахматной анимации во Flutter: руководство по микровзаимодействиям

Flutter'da Aşamalı Animasyon Oluşturma: Mikro Etkileşimler Kılavuzu

Tạo hoạt ảnh so le trong Flutter: Hướng dẫn tương tác vi mô

在 Flutter 中创建交错动画：微交互指南

The Complete Asynchronous Programming Primer for Unity Development

ইউনিটি ডেভেলপমেন্টের জন্য সম্পূর্ণ অ্যাসিঙ্ক্রোনাস প্রোগ্রামিং প্রাইমার

Die vollständige Einführung in die asynchrone Programmierung für die Unity-Entwicklung

El manual completo de programación asíncrona para el desarrollo de Unity

L'introduction complète à la programmation asynchrone pour le développement Unity

एकता विकास के लिए पूर्ण अतुल्यकालिक प्रोग्रामिंग प्राइमर

Unity 開発のための完全な非同期プログラミング入門

Unity 개발을 위한 완벽한 비동기 프로그래밍 입문서

O Primer completo de programação assíncrona para desenvolvimento Unity

Unity Geliştirme için Eksiksiz Eşzamansız Programlama Başlangıç Kılavuzu

Công cụ lập trình không đồng bộ hoàn chỉnh để phát triển Unity

Unity 开发的完整异步编程入门

Clutters and inconsistencies make software development hard. The need for organized task flow and compatibility between a developer's system and production machine goaded the creation and popularity of Docker and other container-based applications like Kubernetes. By the end of this article, you will understand what Docker is, how and when it can be used in the development process, and its overall effect on the software development ecosystem. What is Docker? Docker is an open-source containerizing platform for application services, codes, libraries, configurations, and other dependencies into one package to enhance portability, scalability, and efficiency. It serves as a better alternative to the traditional virtualization of operating systems while running on the host operating system. Docker is written in Go programming language, and the first version was introduced in 2013 as to work on Linux OS. Since then, the name was changed and modified in tandem with subsequent improvements and added features until its current version is available on all popular operating systems. It can now be deployed in cloud services like Microsoft Azure and Amazon web service. dotCloud Here are some terms to get familiar with in understanding Docker: : A Docker image is a read-only immutable file or template needed to execute an application’s code. Inside the Docker image,  a container is built. Docker images : This is a shell document of command lines for defined automation. This is where Docker images are created. Docker file : The Docker container is an environment that includes all the codes and dependencies needed to implement an application. A known feature of the container is its flexibility. Containers Why should developers use it? As a Developer, here are some you should use Docker: : Imagine developing an application with an SQL database without docker; you are likely to encounter test issues with the application on other devices. The conventional route to this problem is to reinstall the OS, but docker eliminates this need — allowing for compatible testings. User experience and compatibility : Using a Docker container, you can share your work with a colleague and test it. Docker allows integrating tools and checkpoints in containers and fixing bugs in the development environment. Debugging capability : Docker is known for its lightweight feature and fast execution of commands. It does not require booting up of an operating system, and it can run workloads with fewer resources that would have been heavier on virtual machines. Development management Conclusion Docker is fast and efficient in handling complex tasks. Big companies like Google, Quora, Spotify, etc., rely on its agile framework. In 2020, a of the most popular technologies by StackOverflow showed that Docker was the third on the list alongside Windows, Android, Mac OS, etc. Soon, Containerization will become a norm and replace the default virtual machine to a recognizable level. survey

A Brief Overview of Docker And Its Impact On The Development Ecosystem

Aladin Ben is an award-winning, transformational gaming executive known for turning startups into successful business ventures. With a proven track record of raising capital and slashing expenses with deep industry connections laser-focused on business objectives, strategic and operational skills, and experience, Aladin reinforces a culture of accountability and a high work ethic. He was one of the core contributors to Dota, co-founder of OG Dota, founding member of a live streaming platform with 10M monthly users, and built Gaimz, a massive social infinite game. Aladin Ben is now investing in a project around user experience and onboarding through social AR and AR worlds called Meekey. Hackernoon got an exclusive interview with the executive, at the Non-fungible Conference in Lisbon, who shared insights on his background and experience. In the beginning, there was a love for gaming Aladin’s journey in gaming started a long time ago. He began as a volunteer contributor in Dota, a community game with no big publisher or a big company that now has over half a million players every month across all the different games. A couple of years later, he started working for an Esports company, ESL, while still managing a Dota community with 1.5 million users, Dotalicious. Afterward, Aladin split his time between venture capital and a live video streaming company, Hitbox, where he was a founding member. Later on, Hitbox was sold, and the entrepreneur co-founded the first Esports team fully player-owner called OG. This Esports team became one of the highest-grossing teams in terms of price in participating in big tournaments, almost $40 million, sponsored by Redbull and with the old players taking the role of coaches. However, his craving for a fully user-generator platform hadn’t been satisfied yet. And that’s where Gaimz comes to play. Building Gaimz Finally, in 2019, moved by his desire to enable people to do what they love and make a living out of it through user-generated paid content, Aladin Ben started working on his own company, Gaimz. His new project used all the knowledge he had from all the companies and games he had worked on before to create a platform to democratize community ownership for content creators, similar to Youtube and Twitch streamers. The main difference between Gaimz and other platforms was the direct access a streamer had to its community while monetizing through interaction and having all their content in one place. So, for example, if one person is playing a video game and live-streaming it, they could invite their fans to play with them, subscribe to the channel, or follow the stream - monetization would come by charging the fans that wanted to play with the streamer. Moreover, Gaimz allows the community to watch Netflix and Youtube, build together and hang out - it’s a social infinite game or metaverse. Unfortunately, Covid-19 hit, and Aladin was forced to sell his project, propelling him to new ventures. Meekey: Making NFTs Accessible According to Aladin Ben, one of the reasons the general public doesn’t get involved with NFTs is that they lack the considerable amount of money required to go through it. At the same time, the Web3 space can grow without more users, which means that projects are sold at a higher price. Spotting this issue, our guest made it his goal to bring users and make it easy to participate in the ecosystem in the most accessible and seamless way. So, he started looking into AR filters on major social platforms, like Instagram, TikTok, and Snapchat, and realized that there were over 1 billion filter users across these platforms. There were 500 million stories posted every day on Instagram using filters. In addition, he found that filter creators weren’t getting a return on their art, meaning their filters weren’t being monetized. Therefore, Meekey’s founder learned that it was possible to bring NFTs and Web3, which were only being discussed on Twitter, to the masses by using filters and Shopify. Thus, Aladin signed creators and reached out to Shopify to create social commerce, where everyone could buy NFTs directly on Instagram or TikTok without the hassle of creating a metamask. In short, people can purchase filters from Meekey and then use these fun and engaging mechanisms in their content and monetize that content. That way, individuals can promote whatever they want on Instagram while participating in the ecosystem. But the perks don’t stop here: We are enabling the seamless and accessible purchase of NFTs. We enable micro-influencers to earn rewards when they share video content using our filters. We enable creators to earn money for creating those AR filters. And we help the whole industry grow in a viral and organic way. - Aladin Ben Upon these developments, the Meekey founder realized one problem: the moment someone posts a filter on one of the social platforms, they stop owning them, and the filter becomes accessible to everyone. So he later developed an application that would allow only the NFT holders to access the filters they’d bought and all the experiences that came with that. Likewise, by using the advertisement budget and giving it to the users that share their experiences in the most viral forms through rewards, you’ll attract more users, and the cycle will continue naturally. Later on, brands will come in to buy a user’s NFT or buy the users’ time, thus scaling up the network and increasing the gains for the parties involved in Meekey. The Future of AR and NFTs Our guest presented us with a fun prospect for the future involving augmented reality and NFTs. Ben’s opinion is that we’ll be able to enjoy AR while outdoors in the future, mixing our old way of life with technology. For example, through smart glasses, someone could be sitting in the park with friends, enjoying the sun, and drinking while having Snoop Dog in AR give them a concert; or go for a walk in New York City and see the Avengers fighting each other instead of simply being in a theater. Hence, there could be different layers to AR and different experiences where one’s NFT would matter, giving people clearance to enjoy distinct adventures. So much more could be said about the future of AR and NFTs and Aladin’s many accomplishments in the gaming world, but it’s probably best to hear it from him. You can find him on and check out more about Meekey . LinkedIn here

Exploring Social AR with Aladin Ben

Explorando la RA social con Aladin Ben

Explorer la RA sociale avec Aladin Ben

अलादीन बेनो के साथ सामाजिक एआर की खोज

Aladin Ben とのソーシャル AR の探索

Explorando AR Social com Aladin Ben

Khám phá mạng xã hội AR với Aladin Ben

与 Aladin Ben 一起探索社交 AR

,  in association with - an innovation management firm, announced - a decentralized storage network, as the winners of the inaugural edition of Polkadot Buildathon: India - Polkadot’s first-ever India-focused blockchain hackathon. Polkadot Lumos Labs Skykiwi As a part of the hackathon, the top three projects stand to receive a grand prize from a pool of $14,000 USD along with go-to-market access and support in technology implementation. The runner-up team was - a gamified music streaming experience using NFTs and the second runner-up team was - an open-source distributed infrastructure for training AI models. PolkaMusic Cerebrum The Buildathon Demo Day The announcement was made at the Polkadot Buildathon Demo Day - a virtual conference with attendees from across the globe on June 11, 2021, where 5 finalists - Cerebrum, Dot ID, Dotscale, PolkaMusic and SkyeKiwi pitched their products one last time. Apart from pitches from the top projects, the Demo day also featured insights into the Polkadot ecosystem, the support offered to developers and entrepreneurs and tech talks from the Web3 Foundation and Parity Technologies teams. Participants were invited to explore a range of opportunities from the Web3 Foundation Grants Program, Substrate Builders Program, On-Chain Treasury, and even an opportunity to secure Job placement in the Polkadot Ecosystem with their knowledge of Substrate. The Polkadot Buildathon: India The surge of emerging advancements in the blockchain industry including DeFi, dApps, NFTs, etc, has resulted in a parallel surge in demand for developers and entrepreneurs in the field. According to a report by , blockchain is the most in-demand hard skill required by top companies in 2020. India with its developer pool at a strong 4.2 million, the second largest globally, is now the hub for blockchain development. Linkedin According to a leading media publication, , the number of blockchain startups in India has also gone up from 100+ in 2018 to 300+ in 2021. This has been a huge pull for global platforms to explore the Indian markets. Inc42 The Polkadot Buildathon: India was established with this mission of building the Web3.0 ecosystem and providing the necessary catalyst to enhance the developer ecosystem of India and be a part of the revolution. The hackathon was conducted in two four-week-long phases: the Ideation phase and the Integration phase. During the course of the Ideation phase, over 1200 developers applied, with 120+ project ideas developed. The participants were provided with training workshops to learn basic local node deployment and identify real-world problems that could be solved on the chain using Substrate, the blockchain-building framework used in the Polkadot ecosystem. This was followed by the Integration phase where the first cohort of the selected participants were empowered to build and improve their solutions on the Polkadot parachain, fine-tuning their USPs. said . “This has been a very exciting and rewarding experience and we wish the winners the best for their future endeavours. At the Web3 Foundation, we are delighted to have been a part of such  a dynamic and strong developer community in India and look forward to working on more future projects with the community filled with the same enthusiasm, knowledge, and vigour as we witnessed throughout the Buildathon," , Urban Osvald, Growth Manager at Web 3.0 Foundation He also added, "As the world heads towards a shift into Web3, India has proved to be abreast with it, if not ahead. Our Polkadot Buildathon:India was a step into empowering the community further and enabling them to explore the limits of the partnership between their creativity and this disruptive technology. After witnessing such a spectacular turnout, we are very excited to pursue future collaborations and see what more Indian developers, entrepreneurs, innovators, startups, etc have to offer." The Polkadot Buildathon: India hackathon saw a strong and active participation of over 1200+ registrations from which top participants selected in its first cohort were granted $200 each, followed by the selection of the Top 5 who pitched their projects live at the Demo Day. The 8-week long hackathon was launched in partnership with , an innovation management firm specialising in running technology open innovation programs in India. Lumos Labs

Meet the winners of the Polkadot Buildathon: India

Have you ever had to find unique topics in a set of documents? If you have, then you’ve probably worked with Latent Dirichlet Allocation (LDA). This is how LDA works: The algorithm searches for the same word clusters across the documents you're working with and generates any so-called unique topics. These are topics that occur at some frequency in the documents. LDA is widely used for online content generation. This algorithm is particularly helpful in sales and marketing. Suppose that you want to create a logo for your company. How can you find the one that will best represent your business and attract clients? The answer is to study the existing cases and review the competitors’ strategies. To create an effective logo, you should gather data from competitors’ websites. That is, . Based on the results you get, you will be able to outline the keywords most frequently associated with your sphere and create an attractive logo. you need to parse websites and apply LDA to generate goal-related topic clusters for you Or suppose that you want to optimize your content based on SEO research. . Based on the results, you can adjust your content to make it semantically valuable for search engines. Applying LDA will allow you to gather unique topic clusters with keywords in each cluster You can use LDA not only for websites but for any type of textual data. So, if you want to use LDA to generate unique topics, you need to have a large number of documents or websites to use for your analysis. But that’s not all: . Stop words are the words that bear no value in the analysis, like “the,” “your,” “him,” “a,” and so on. to get accurate results, you have to remove stop words from the dataset first Apart from deleting stop words, you have to delete stuff related to HTML, CSS, JavaSript SEO, and other words not related to LDA analysis. Does that seem too complicated? Let’s lay it all out. In this article, we will discuss how to generate unique topics with LDA when parsing websites. How to Start Using the LDA Algorithm To parse websites with LDA, first, you need to import the necessary libraries for file processing and cleaning the HTML, CSS, JavaScript text. You'll also import the LDA analyzer: os re gensim nltk time random itertools pandas pd string digits wordcloud WordCloud gensim.utils simple_preprocess nltk.corpus stopwords nltk.stem.porter PorterStemmer tqdm.notebook tqdm, trange gensim.corpora corpora bs4 BeautifulSoup import import import import import import import import as from import from import from import from import from import from import import as from import Next, select the number of websites you need to analyze. The larger and more precise your sample is, the more correct the LDA results will be. Here, both the quantity and the quality of the sample matter. That is, you should create a uniform sample of websites that may contain the necessary keywords for a topic generation. Then, choose the folder that will be used to read the documents: entries = os.listdir( ) './dataset_htmls/' Next, read the chosen websites and conduct a preliminary cleaning of the HTML, CSS, JavaScript: def cleanDocument(html):
    soup = BeautifulSoup(html, ) script soup([ , ]): 
        script.extract()

    text = soup.get_text()
    lines = (line.strip() line text.splitlines())
    chunks = (phrase.strip() line lines phrase line.split( ))
    text = .join(chunk chunk chunks chunk) text


start_time = time.time()

files = [ ] * files_number 
df = pd.DataFrame(data={}, columns = [ ]) i range(files_number): open( + entries[i], , encoding= ) f: :
            text = cleanDocument(f.read())
            text = re.sub(r , , text)
            df = df.append({ : text}, ignore_index=True)   
            
        except:
            files[i] = print( + entries[i])
            
print( % (time.time() - start_time)) "html.parser" for in "script" "style" for in for in for in "  " '\n' for in if return 0 'text' for in with "./dataset_htmls/" "r" 'utf-8' as try '\b\w{1,2}\b' '' 'text' " " "problem with file" "--- %s seconds ---" When you're using LDA, keep in mind that there are a large number of frameworks for creating websites like Angular, Vue, React, WordPress, and more. Also, there are a lot of ready-made website builders that allow you to build websites using a no-code technique like Wix or Tilda. And each developer uses their own best practices to build websites. The issue is that most problems occur because of the so-called "dirty hacks" that developers use. With these dirty hacks, websites bypass the parser and analyzer which poses a problem for the LDA. How the LDA Algorithm Works Let’s start with CSS. Make sure you detect the following ways of defining styles: Inline CSS — use of style attribute in HTML tags Inline CSS in the head section — use of in the document header <style> External CSS like a file — use of to load a CSS file <link> For HTML, it's important to delete tags but not to delete the content of these tags. The same goes for JavaScript. One way to do so is to use regular expressions: df[ ] = df[ ].map(lambda x: re.sub( , , x))
df[ ] = df[ ].map(lambda x: re.sub( , , x))
df[ ] = df[ ].map(lambda x: re.sub(r , , x, flags=re.DOTALL)) 'text' 'text' '<[^<]+?>' '' 'text' 'text' '(?s)<style>(.*?)<\/style>' '' 'text' 'text' '<script.+?</script>' '' Yet, there is a problem with such methods. Using single tags, JavaScript plugins, and various ways of building web applications means that you cannot completely remove HTML, CSS, and JavaScript. But you can easily check this in a site's input files. This is why, for writing this article, I used the bs4 library. To test the cleanup, I chose 1,000 websites where no significant bugs with removing web elements were detected. So you also can use this library and remove web elements that are not related to the LDA analysis via the function. cleanDocument The next step is to delete complex words consisting of two words. Use the following regular expression to solve this problem: text = re.sub(r , , text) '\b\w{1,2}\b' '' All cleaned text is recorded to dataframes. To evaluate the work of the algorithm, look at the running time. The running time for 100 websites is about 5 seconds. df Your next task is clean for stop words. As I mentioned, stop words refer to the words that bear no value for analysis like “the,” “you,” “himself,” and so on. To clean stop words, download a list of standard words for the English language. Apart from stop words, consider the standard syntax used on nearly every website. For example, such words as “contact us,” “cookies,” “confirm,” “back,” and so on. Then add those words to your list of stop words. Also, when cleaning, review the "standard" set of words used in SEO optimization. other_stop_words = [ , , , , , 
]
    nltk.download( )
stop_words = stopwords.words( )
stop_words.extend(other_stop_words)

def sent_to_words(sentences): sentence sentences: (gensim.utils.simple_preprocess(str(sentence), deacc=True))
        
def remove_stopwords(texts): [[word word simple_preprocess(str(doc)) word not stop_words] doc texts]

data = df[ ].values.tolist()
data_words = list(sent_to_words(data))

data_words = remove_stopwords(data_words) 'contact' 'cookies' 'confirm' 'website' 'share' 'stopwords' 'english' for in yield return for in if in for in 'text' Congratulations! You are halfway there. What’s next? You need to form a word dictionary for the LDA algorithm: id2word = corpora.Dictionary(data_words)
texts = data_words
corpus = [id2word.doc2bow(text) text texts]

num_topics = lda_model = gensim.models.LdaMulticore(corpus=corpus,
                                       id2word=id2word,
                                       num_topics=num_topics)

print(lda_model.print_topics()) for in 10 Important note: to get precise results, you need to make a dictionary of stop words that will be used to clean the text. To see the output of this procedure, use the frequency counting of words in all text. This will let you determine which words occur the most frequently: def freq(data_words):
    df = pd.DataFrame({}, columns=[ , ])
    
    unique_words = set(data_words) word unique_words:
        df = df.append({ : word, : data_words.count(word)}, 
                ignore_index=True) df

data_words = itertools.chain.from_iterable(data_words)
data_words = list(data_words)
frequency = freq(data_words)
frequency = frequency.sort_values(by= , ascending=False)
frequency.to_csv( ) 'Word' 'Frequency' for in 'Word' 'Frequency' return "Frequency" 'frequency.csv' The Results of LDA Analysis Let’s review the results. We'll use as an example 10 websites that provide courses for learning programming languages. Based on the algorithm above, I parsed the 10 websites and removed all words unrelated to the analysis. You can see the first result of the LDA analysis in the following image: Below, you can see the results of frequency analysis: Frequency analysis provides information for SEO optimization where words can be used as search keys. However, to find a cluster of similar and "valuable" words, you should clean the text of base words. In our case, base words include: , , , , , , , , , , , , , , , , , , , , 'course' 'learn' 'programming' 'lpa' 'code' 'courses' 'java' 'learning' 'get' 'one' 'python' 'see' 'pluralsight' 'web' 'org' 'become' 'javascript' 'android' 'may' 'site' 'developer.' You can identify these words based on frequency analysis or the results of the LDA algorithm after iterations. n After cleaning out the base words we get the following result: In fact, the output result is a set of topics with the following set of words: Team, students, read, review, free, skills, reviews, path, data, project, build, masterclass, instructor, job, etc. Keep in mind that you need to clean for topics manually and iteratively until you get your desired results. In such algorithms, you get only the roots of words. The endings, plural forms, and duplicated words will be rejected. That’s it — if you are interested in more info, I welcome you to check out the . full code Wrapping Up You can see that LDA can be an effective way of topic modeling. Based on the results above, you can easily choose the right word combination for your logo. You can also optimize your existing content and adjust it to perform better based on the results. The advantage of LDA lies in the fact that its functions are flexible. Its purpose is primarily mathematical, so you can adjust LDA’s work to your needs. The disadvantage, though, is that you need to polish the results manually. While this is more time-consuming, you get more precise results in the end. So if you deal with marketing and SEO, using LDA is a perfect way to dive deeper into content generation. . Special thanks for co-authoring this article to Volodia Andrushchak, Machine Learning Engineer at @KeenEthics If you are interested in other useful readings on Internet of Things and Artificial Intelligence, check other articles by Volodia: https://keenethics.com/volodya-andrushchak?article=2697 Thank you for reading, and I sincerely hope that you found it helpful!

Using the LDA Algorithm for Websites

We've been working recently on adding rules to help write better regular expressions in Java. I've talked already about rules to and with . Those rules help you make sure your regular expressions are accurate and maintainable. Today I will show you how to make sure that the regular expression won’t crash your program. find errors in character classes the use of boundary markers and overly complex regular expressions As a brief reminder: regular expressions are a terse and powerful mechanism for matching patterns in strings. Part of the power of regular expressions is that a pattern can concisely describe a string that might be much larger than the original pattern. Sometimes much, much larger. Sometimes enough to overflow the stack and crash your application. Due to the way regular expression matching is implemented in Java (and many other languages/libraries), matching a pattern may - depending on the regex - require stack space proportional to the length of the input. This means large inputs could cause the program to crash with a when you try to use the regex. StackOverflowException We ran into this problem in our own code when analysis tried  to find comments containing at least one word with more than a given number of characters. For this we used a regex similar to , which would cause the analysis to crash on source files containing sufficiently long comments. To prevent problems like this for our future selves as well as for you, dear users, we implemented a rule which can detect problems like this: #(.|\n)*\w{3,} java:S5998 - Regular expressions should not overflow the stack In theory any regex is susceptible to stack overflow if it contains a repetition that contains some sort of branching, such as another repetition or an alternative. However, for some regular expressions the length of input required to make them crash is much larger than for others. To account for this, our rule tries to estimate how much stack space a regular expression will consume relative to the input size and only raises issues on regular expressions whose stack consumption exceeds a configurable threshold. One way to avoid stack consumption in regular expressions is to use . Possessive quantifiers are created by adding a to a quantifier (e.g. instead of ). Doing so disables . In addition to avoiding issues with catastrophic backtracking, making a quantifier possessive allows a pattern to be matched without consuming stack space. The problem is that sometimes backtracking is necessary and using possessive quantifiers to disable it may leave you with a regex that can never match any input. possessive quantifiers + x*+ x* backtracking Consider for example the regex I cited above: : If we try to fix the stack overflow by making the quantifier possessive, we end up with . This regex can never match anything because any input that could be matched by will already have been matched by and, being possessive, it won’t give it back. Luckily we have a rule that warns you about issues like this: #(.|\n)*\w{3,} #(.|\n)*+\w{3,} \w (.|\n)*+ java:S5994 - Regex patterns following a possessive quantifier should not always fail So how would one properly fix this issue? In our case, we decided to get rid of the regex altogether since there wasn’t really a good reason to use a regex here. But the regex could also be salvaged easily enough by getting rid of the alternation. The intent of was to match any character, including line breaks, because . does not match line breaks by default. The proper way to match any character would be to enable the flag, which can also be enabled from within the regex using . So either or would work (the latter of which also works when using methods that don’t involve ). .|\n DOTALL (?s) Pattern.compile(“#.*\\w{3,}”, Pattern.DOTALL) ”(?s)#.*\\{3,}” Pattern.compile Another regex susceptible to stack overflows would be something as simple as . Here the alternation can be removed by using a character class instead, which doesn’t require stack space to match: . Luckily we have a rule that finds alternations that can be replaced by character classes and suggests just that replacement like in the following code: (a|b)* [ab]* Spring Boot java:S6035 - Single-character alternations in regular expressions should be replaced with character classes Regular expressions are a truly powerful feature. Because great power comes with great responsibility (and great opportunities for screwing things up, or confusing your colleagues and future-you), we now offer a total of 24 rules targeting regular expressions. They are all available today in SonarCloud. These three rules around avoiding stack overflows are available starting from SonarQube 8.7, but the rest are already released. You can see them all at . rules.sonarsource.com Finally, note that some of the rules perform in-depth automata-based analyses on your regular expressions to identify issues as accurately as possible. This is an extremely promising approach and we will probably discuss this feature in a future blog post because we truly believe it’s amazing. This will for sure bring A LOT to regex code quality in general, so stay tuned!

Avoid Stack Overflows By Writing Regexes Properly

We software engineers have a superpower. It’s true! We do. And you came here to find out what your superpower is. How long do I keep you in suspense? One sentence? Two? Or do we just cut the chase. Let the cat out of the bag. Well okay, I’ll tell you. In a moment. First, I want to show you a number: 16% I was astounded when I saw this number. I was working for a software engineering department that very generously granted all engineers a sizeable personal training budget. You could spend it on anything related to personal education that wasn’t a paper book (for tax reasons…). It was more then enough for a ticket to a decent conference, a workshop, or a virtually limitless supply of online courses. And it wasn’t being spent. 16% was the amount of the overall training budget that had been spent towards the end of the financial year. Our company was to make us more awesome, and we weren’t spending it. giving us money This, I thought, was really quite unfortunate — because our superpower is learning. Yes, learning. I wanted to know why my colleagues were not spending their training budget. The people I was working with were all highly intelligent and incredibly talented. Absolutely some of the best engineers I have ever worked with. Why weren’t they spending their training budget? I conducted an informal survey and a few common themes emerged: I have no time I don’t know what to spend it on I already know what I need to know for my job. I know I should learn more stuff, but it’s hard to be motivated when I feel like I won’t get a chance to apply them in my job. As a father of two — time is definitely a legitimate excuse. There’s not much we can do about that — although in a future article I’ll share my personal tips for learning efficiently when you have no time. You may or may not be lucky enough to have access to a training budget or a reimbursement program. That’s okay. There’s plenty you can learn just by spending an hour a day reading . But setting aside the ‘how’, in this essay I’d like to convince you of two things: Hacker Noon That acquiring broad technical knowledge is worthwhile even if you think you can’t directly apply it in your current role That learning really is a superpower I’m going to do this by addressing four key themes, which I’ll attempt to stitch together at the end. Hopefully with a dazzling display of logic and reason. If not, then with a lot of exclamation points and another ‘kapow!’. 1. True radical and disruptive innovation are rare To many of us the word ‘innovation’ is synonymous with ‘disruption’. But that is not solely the case. In the late 1980’s and early 90’s researchers Henderson and Clark proposed the ‘ ’ of innovation. They were looking to explain why, in some cases, relatively minor improvements in technology had disastrous effects on industry incumbents. architectural model The example from their research paper was Xerox, who had been dominating the plain paper copier market in the 1970s and in fact had invented the core technologies. However when competitors started offering smaller, more reliable copiers it took Xerox eight years to launch a competitive offering. In that time it lost about half it’s market share. In the case of Xerox the actual underlying technological improvements were relatively minor. The core concepts were unchanged. We won’t talk about the model of innovation itself in this essay, but rather the consequences. It’s informative to consider how market leaders can crumble. How previously cutting-edge, innovative companies can be left behind. Did they have bad management? Did they rest on their laurels and stop innovating? Did they ignore their customers? Well, no. In many cases market leaders were doing everything right. They had good management, they were listening to their customers. An example of this is Kodak. When digital cameras first arrived on the scene they were slow, fragile, took terrible photos and held less images than a film roll. I recall my uncle had an early model digital camera which he used to take photos for his product catalogue. It saved photos onto a 1.44" floppy. Why would Kodak invest in a worse product when customers want better film? Kodak did invest in digital cameras, but made the mistake of trying to match the quality of their film. Early technology was not up to the task and they failed to gain a firm foothold. They also anticipated that online photo sharing would rise dramatically. Fatally, they hedged a bet that people would share photos online in order to print them. They failed to realize that online sharing was a new business model, . not a way to sell more film As Clayton Christensen wrote in his book , you can find yourself becoming incrementally better at something that people are wanting less and less. When this happens you can’t innovate your product. But you can innovate your business model. Kodak was one example, but I have two more that are closer to the heart of us engineers. Docker and AWS EC2. The Inventors Dilemma The story behind Docker is an interesting one. The company behind Docker was originally ‘dotCloud’, who were making some kind of polyglot PaaS thing that no-one really cared about. They were just about to go under when they had the insight that the tools they were building their product with were more interesting than the product itself. So they pivoted to try and build a business model around Docker, although when they started they had no idea what that business model would be. But Docker is nothing new! It is built on top of cgroups which were added to the Linux kernel in 2007. Before cgroups were in Linux they were in Unix. We had Solaris Zones around 2004. But before we had containers in Unix they were in Linux — we had VSERVER around 2001. Although they were in Unix before that, with FreeBSD ‘jails’. Ultimately though it all boils down to the system call that dates back to 1979. chroot Docker is not new! EC2, my second example, was certainly nothing new from a technological standpoint. Just a virtual machine behind a REST API. What’s new about that? Nothing except the pricing model. When we boiled docker we got chroot. When we boil EC2 we get virtual memory and RPC. EC2 was incredibly disruptive, but not technically. Under the hood the technology was familiar, only an incremental improvement on what had come before. This essay is meant to be about learning, and here I’ve been rabbiting on about innovation. The point I’m trying to make is that even though it may feel at times that technology evolves at a blinding pace, the underlying technology moves more slowly than it appears on the surface. True disruptive innovation absolutely happens, but most disruption actually happens when tiny, incremental improvements to existing technology reach a tipping point, or someone changes the business model. So, at last, we reach the pointy end of the point. The “oh that’s interesting but I can’t apply it in my job” technology you see today could be tomorrow’s disrupter. Broad learning now could give you a leg-up when that next big ‘paradigm shift’ comes along. 2. Learning accelerates learning The corollary to the previous point is that new technology is mostly just old technology with a version bump and a new logo. Which means that the more you know, the easier it is to assimilate new knowledge. Learning will accelerate learning. The more you know, the more you know! Two illustrate the point let’s do a simplified breakdown of two technologies that can be easily understood in relation to their ancestors or inspiration. EC2 and Kotlin. Simplified lineage of EC2 As we mentioned already, EC2 is just a virtual machine with a REST API. Virtual machines are build on the concepts of virtual memory — while down the other fork we see REST is like SOAP but with JSON, and SOAP itself was an RPC mechanic like CORBA, RMI or DCOM before it, but over HTTP. This is a vast over-simplification of course, but I feel it serves to illustrate the point. Our next example is a programming languages. One of my favorites, Kotlin. Simplified lineage of Kotlin In the diagram above I’ve attempted to illustrate the evolution of Kotlin in terms of the languages that most directly inspired it and the languages that inspired them in turn. (This may not be historically accurate, but it’s how I think of it anyway). As I’m fond of telling anyone who wanders too close (even strangers in the supermarket, they hate that!), “Kotlin is just C# with Scala syntax for the JVM”. If you happen to know C# and Scala, you know Kotlin. already So learning new stuff is easy, if you already know everything else. Although, admittedly, that’s a lot to learn. Especially if you don’t already know everything else. But there’s good news! You don’t actually need all of that knowledge in your own head to gain benefits from it. Which is my segue into a seemingly unrelated topic: diversity. 3. There is value in diversity Having a diverse and inclusive culture is simply the decent and ethical human thing to do. No arguments. But curious souls have sought to quantify if there might be other benefits as well, and it turns out there are! According to a and the Credit Suisse Research Institute: 2015 McKinsey report Diverse companies are 35% more likely to have higher financial returns Gender diverse companies are 15% more likely to have higher financial returns Companies with lower diversity are less likely to have higher financial returns Why? How does the diversity effect work? How does it improve your company’s bottom line? Some known benefits that can improve profitability are that diversity helps you win the war on talent; strengthen customer orientation; increase employee satisfaction; enhance the company’s image and improve decision making. It’s this last point we’re going to zoom in on. Diversity helps you improve decision making. Why? I’m not sure anyone can say for certain, but in the words of one CEO: “Diversity creates dissent, and you need that. Without it, you’re not going to get any deep inquiry or breakthroughs” The current thinking is that people from diverse backgrounds are more likely to challenge each other. If this is done constructively it leads to multiple perspectives and debate, which in turn create a situation in which the participants are more likely to detect novel solutions. Intriguingly, just as there are different types of innovation there are also different types of diversity. The main types are: Social-category diversity —such as ethnicity, gender, age and physical disabilities Informational diversity — such as education, experience, tenure, skills, expertise and functional knowledge. Value diversity — such as ideological beliefs, values and personal differences This being an essay on learning, we’re going to focus on informational diversity. This alone is not enough to create the benefits of a diverse and inclusive team, but it one aspect of the whole that you can address simply and effectively. As a software-engineer you’ve probably felt what a lack of informational diversity feels like. It’s that uphill battle you face when you suggest bringing in a new language, trying a new architectural style, changing your logging format, and such as that. It’s the grumbling, the push-back, the ‘Java 7 has always been good enough for us’. It’s not where you want to be. Skepticism is fine. is proven technology. Boring technology But “prove it” is fine. That’s a great response if it indicates a willingness to be convinced. Much better than “No. That’s not how we do it here”. It’s also better than, “Yeah, sure.” Because you might be wrong, and you might have just led the team down a six-month rabbit hole trying to justify why a gossip-based consensus protocol that requires mesh connectivity is worth drilling holes through your firewall for. Not that I’m bitter or anything. Curiously, the way you arrange your diverse teams actually makes a difference. Let’s say you have 100 engineers in your department but only ten of them are women. You want the benefits of diversity, so what should you do? Ten teams of ten, each with one woman? In the picture below I show the different types of diverse teams. Before moving on to the next paragraph, challenge yourself to think for a moment about which type of team might best exhibit the benefits of a diverse team. Types of diverse teams A uniform team is clearly not diverse, we can rule that one out. A skewed team as you might imagine contains only a token minority, and these tend to not perform so well. In studies so far, tilted teams actually seem to show more of the benefits of diversity than balanced teams. Perhaps because a balance reduces some of the dissent and constructive criticism? This does mean, however, that as your company works towards improving diversity you may not be in a position to have all your your teams be diverse. What can you do with uniform teams? Are they doomed to play second fiddle to your creative, high-achieving diverse-teams? Actually no. It turns out that uniform teams have strengths of their own. Uniform teams seem to perform better at ‘exploitation’ tasks. That is, work involving a high degree of production, efficiency and execution. Uniform teams are great at churning the work out. Diverse teams, on the other hand, perform better at work requiring experimentation, innovation and problem solving. Now … what sort of work would you prefer? This is completely and totally a personal opinion, but my preference is definitely for the second type of team. If yours is too, you want to be part of a diverse team, and as we have seen, diversity of thought absolutely counts. Moreover, diversity of thought is something you can improve through continuous, broad learning. Even better, you don’t all have to learn the same things. In fact it contributes more to diversity if your team all learn different things and acquire different skills. 4. Break out of your comfort zone In 2015 I attended the conference by accident. I bumped into one of the engineering leads in the hallway (actually it wasn’t a hallway because open plan — but it was the place the hallway would have been if we had had hallways). YOW Sydney “Oh hai”, he said. (I’m paraphrasing). “Would you like to go to YOW tomorrow? We’re exhibiting and I’m meant to be manning the booth, but I need to attend an all-day board meeting instead.” “Umm,” I reply, “let me just delete all my calendar appointments… I mean think about it… YES!” So I went, and one of the was by Dave Thomas, author of ‘ ’, which us engineers of-a-certain-age will know well. I can summarize his talk thus: talks Pragmatic Programmer Dave Thomas is an old fart Old farts get stuck in their ways So do young ones You get comfortable, you get lazy What do you call a lazy programmer? Okay — what do you call a lazy programmer? You can derive the answer by looking at the previous section on diversity. What are uniform teams good at? Production, efficiency, execution. Without creativity, innovation or debate. If you’re stuck in a rut, working on the same code-base, doing the same old thing day in and day out, there’s a name for that. To me it sounds like we’re describing a … Maintenance Programmer Bring it home now! So to summarize, learning a super-power, because: is Disruptive technologies will not catch you off guard. You saw them coming! You will assimilate new knowledge at lightning pace! You can combine powers with your colleagues and let the innovation flow! You will not get stuck in a rut. You will stay passionate, find creative solutions and enjoy your work! Kapow! Image source: Pixabay

You Have A Superpower

“You don’t need a silver fork to eat good food.” 🎬 Introduction Wine Reviews In this article, I will try to explore the Wine Reviews Dataset. It contains 130k of reviews in Wine Reviews. And at the end of this article, I will try to make simple text summarizer that will summarize given reviews. The summarized reviews can be used as a reviews title also.I will use spaCy as natural language processing library for handling this project. 📋 Object Of This Project The objective of this project is to build a model that can create relevant summaries for reviews written on Wine reviews. This dataset contains above 130k reviews, and is hosted on . Kaggle What Is Text Summarization? Text summarization is the process of distilling the most important information from a source (or sources) to produce an abridged version for a particular user (or users) and task (or tasks). Why we need text summarization? Imgur In Recent Years we are witnessing the amount of textual information is increasing day by day .The Textual Information grows rapidly. It becomes more difficult for the user to read the textual information and also it leads to loss of interest. That is the reason why Text Summarization came into picture which will solve this problem. Types of Text Summarization Methods Text summarization methods can be classified into different types. i. Based on input type: Single Document, where the input length is short. Many of the early summarization systems dealt with single document summarization. Multi Document, where the input can be arbitrarily long. ii. Based on the purpose: Generic, where the model makes no assumptions about the domain or content of the text to be summarized and treats all inputs as homogeneous. The majority of the work that has been done revolves around generic summarization. Domain-specific, where the model uses domain-specific knowledge to form a more accurate summary. For example, summarizing research papers of a specific domain, biomedical documents, etc. Query-based, where the summary only contains information which answers natural language questions about the input text. iii. Based on output type: Extractive, where important sentences are selected from the input text to form a summary. Most summarization approaches today are extractive in nature. Abstractive, where the model forms its own phrases and sentences to offer a more coherent summary, like what a human would generate. This approach is definitely a more appealing, but much more difficult than extractive summarization. Prerequisites This article makes the following assumptions: You are familar with Python You have Python 3.6 or greater installed on your system spaCy package. What is spaCy? spaCy is a relatively new package for “Industrial strength NLP in Python” developed by Matt Honnibal at . It is designed with the applied data scientist in mind, meaning it does not weigh the user down with decisions over what esoteric algorithms to use for common tasks and it’s fast. Incredibly fast (it’s implemented in Cython). If you are familiar with the Python data science stack, spaCy is your for NLP – it’s reasonably low-level, but very intuitive and performant.However, since SpaCy is a relative new NLP library, and it’s not as widely adopted as NLTK. explosion.ai numpy Installation of spaCy , its data, and its models can be easily installed using python package index and setup tools. Use the following command to install spacy in your machine: **spaCy** ! pip install spacy In case of Python3, replace “pip” with “pip3” in the above command. OR download the source from and run the following command, after unzipping: here !python setup.py install To download all the data and models, run the following command, after the installation: !python -m spacy.en.download all You are now all set to explore and use spacy. Loading spaCy Libraries import spacy Implementation Section 1. Import Packages import numpy as np import spacynlp = spacy.load('en_core_web_sm')import pandas as pd import seaborn as snsimport matplotlib.pyplot as pltfrom wordcloud import WordCloudimport stringimport refrom collections import Counterfrom time import time from nltk.corpus import stopwordsimport nltkimport plotly.offline as pyimport plotly.graph_objs as goimport plotly.tools as tls%matplotlib inline # linear algebra # data processing, CSV file I/O (e.g. pd.read_csv) # from sklearn.feature_extraction.stop_words import ENGLISH_STOP_WORDS as stopwords stopwords = stopwords.words('english')sns.set_context('notebook') 2. Import Dataset In this section, I will load the desired dataset for this notebook. This dataset has huge number of reviews. It will be hard to work with full dataset. So I will randomly sample the dataset into smaller chunks for easy purpose. reviews = pd.read_csv("../input/winemag-data-130k-v2.csv", nrows=5000,usecols =['points', 'title', 'description'],encoding='latin1')reviews = reviews.dropna()reviews.head(15) 3. Text preprocessing In this step, I will be using Spacy for preprocessing text, in others words I will clearing not useful features from reviews title like punctuation, stopwords. For this task, there are two useful libraries available in Python. 1. NLTK 2. Spacy. In this notebook, I will be working with Spacy because it is very fast and has many useful features compared to NLTK. So without further do let’s get started! !python -m spacy download en_core_web_lgnlp = spacy.load('en_core_web_lg')def normalize_text(text):tm1 = re.sub('<pre>.*?</pre>', '', text, flags=re.DOTALL)tm2 = re.sub('<code>.*?</code>', '', tm1, flags=re.DOTALL)tm3 = re.sub('<[^>]+>©', '', tm1, flags=re.DOTALL)return tm3.replace(" ", "") \n Output screen: reviews['description_Cleaned_1'] = reviews['description'].apply(normalize_text) # in this step we are going to remove code syntax from text print('Before normalizing text----- ')print(reviews['description'][2])print(' After normalizing text----- ')print(reviews['description_Cleaned_1'][2]) \n \n \n Output screen: We can see a huge difference after normalizing our text. Now we can see our text is more manageable. This will help us to explore the reviews and later making summarizer. We are also seeing that there are some punctuation and stopwords. We also don’t need them. In the first place, I don’t remove them because we are gonna need this in future when we will make summarizer. So let’s make another column that will store our normalized text without punctuation and stopwords. 3.1 Clean text before feeding it to spaCy punctuations = '!"#$%& ()*+,-/:;<=>?@[ ]^_`{|}~©' def cleanup_text(docs, logging=False):texts = []doc = nlp(docs, disable=['parser', 'ner'])tokens = [tok.lemma_.lower().strip() for tok doc if tok.lemma_ != '-PRON-']tokens = [tok for tok tokens if tok stopwords tok punctuations]tokens = ' '.join(tokens)texts.append(tokens)return pd.Series(texts)reviews['Description_Cleaned'] = reviews['description_Cleaned_1'].apply(lambda x: cleanup_text(x, False)) \' \\ # Define function to cleanup text by removing personal pronouns, stopwords, and puncuation in in not in and not in print('Reviews description with punctuatin and stopwords--- ')print(reviews['description_Cleaned_1'][0])print(' Reviews description after removing punctuation and stopwrods--- ')print(reviews['Description_Cleaned'][0]) \n \n \n Output screen: Wow! See! Now our text looks much readable and less messy! 4. Distribution of Points In this section, I will try understand the distribution of points. Here points mean number of upvote the description got in social media(such as facebook,twitter etc). plt.subplot(1, 2, 1)(reviews['points']).plot.hist(bins=30, figsize=(30,5), edgecolor='white',range=[0,150])plt.xlabel('Number of points', fontsize=17)plt.ylabel('frequency', fontsize=17)plt.tick_params(labelsize=15)plt.title('Number of points description', fontsize=17)plt.show() Output screen: The description of points lies between 80 to 100 mostly. Majority of the description got points between 80 to 100. 5. Analyze reviews description In this section, I will try to analyze wine description. In Wine Reviews, the wine description plays a vital role. A good description can make your wine stand out. It also helps get a reviews faster. Lastly, It will help you get some points. Let’s see what we can find in the wine description. reviews['Title_len'] = reviews['Description_Cleaned'].str.split().str.len()rev = reviews.groupby('Title_len')['points'].mean().reset_index()trace1 = go.Scatter(x = rev['Title_len'],y = rev['points'],mode = 'lines+markers',name = 'lines+markers')layout = dict(title= 'Average points by wine description Length',yaxis = dict(title='Average points'),xaxis = dict(title='wine description Length'))fig=dict(data=[trace1], layout=layout)py.iplot(fig) Output screen: 6. Description Summarizer Siraj Raval In this step, I will try to make a description summarizer. There is a huge amount of research going for text summarization. But I will try to do a simple technique for text summarization. The technique describes below. 6.1 Convert Paragraphs to Sentences We first need to convert the whole paragraph into sentences. The most common way of converting paragraphs to sentences is to split the paragraph whenever a period is encountered. 6.2 Text Preprocessing After converting paragraph to sentences, we need to remove all the special characters, stop words and numbers from all the sentences. 6.3 Tokenizing the Sentences We need to tokenize all the sentences to get all the words that exist in the sentences 6.4 Find Weighted Frequency of Occurrence Next we need to find the weighted frequency of occurrences of all the words. We can find the weighted frequency of each word by dividing its frequency by the frequency of the most occurring word. 6.5 Replace Words by Weighted Frequency in Original Sentences The final step is to plug the weighted frequency in place of the corresponding words in original sentences and finding their sum. It is important to mention that weighted frequency for the words removed during preprocessing (stop words, punctuation, digits etc.) will be zero and therefore is not required to be added 6.6 Sort Sentences in Descending Order of Sum The final step is to sort the sentences in inverse order of their sum. The sentences with highest frequencies summarize the text. Function for text summarization: This function help for summarization from big text.So we need this function all time when we want to summarization from text.The function below here: def generate_summary(text_without_removing_dot, cleaned_text):sample_text = text_without_removing_dotdoc = nlp(sample_text)sentence_list=[]for idx, sentence enumerate(doc.sents): sentence_list.append(re.sub(r'[^\w\s]','',str(sentence))) in # we are using spacy for sentence tokenization stopwords = nltk.corpus.stopwords.words('english')  

word\_frequencies = {}    
for word **in** nltk.word\_tokenize(cleaned\_text):    
    if word **not** **in** stopwords:  
        if word **not** **in** word\_frequencies.keys():  
            word\_frequencies\[word\] = 1  
        else:  
            word\_frequencies\[word\] += 1  


maximum\_frequncy = max(word\_frequencies.values())  

for word **in** word\_frequencies.keys():    
    word\_frequencies\[word\] = (word\_frequencies\[word\]/maximum\_frequncy)  


sentence\_scores = {}    
for sent **in** sentence\_list:    
    for word **in** nltk.word\_tokenize(sent.lower()):  
        if word **in** word\_frequencies.keys():  
            if len(sent.split(' ')) < 30:  
                if sent **not** **in** sentence\_scores.keys():  
                    sentence\_scores\[sent\] = word\_frequencies\[word\]  
                else:  
                    sentence\_scores\[sent\] += word\_frequencies\[word\]  


summary\_sentences = heapq.nlargest(7, sentence\_scores, key=sentence\_scores.get)  

summary = ' '.join(summary\_sentences)  
print("Original Text:**\\n**")  
print(text\_without\_removing\_dot)  
print('**\\n\\n**Summarized text:**\\n**')  
print(summary) Now we have written the function let’s try to summarize some descriptions. generate_summary(reviews['description_Cleaned_1'][8], reviews['Description_Cleaned'][8]) Output screen: generate_summary(reviews['description_Cleaned_1'][100], reviews['Description_Cleaned'][100]) Output screen: generate_summary(reviews['description_Cleaned_1'][500], reviews['Description_Cleaned'][500]) Output screen: That’s awesome! We successfully made a simple winemag description summarizer. 7. Conclusion Thanks for reading this article. If you have any suggestion feel free to reach me in the comment or sent or connect on . Stay in touch for more update. Thank you. 😎 mail LinkedIn For the full code visit . Kaggle “Let us celebrate the occasion with wine and sweet words.” If you like this article then give 👏 clap. Happy Analysis!

📚 Summarization With Wine Reviews Using spaCy📋

Generating and Decoding QRCodes using Python modules from Elixir app The Erlang VM, on which Elixir runs, is efficient for applications in particular domains but falls short when it comes to other operations like data . Python on the other hand, has a rich set of packages for tackling data processing and scientific calculations. processing and computation-heavy stuff Luckily the Erlang VM allows interoperability with other languages using the . Erlang port protocol is a library for Elixir which makes it easier to connect Erlang to a number of other programming languages using the Erlang port protocol. Currently Erlport supports and . ErlPort Python Ruby In this post we’ll see how we can call Python module to generate QRCode right from our Elixir app. **Erlport Python Primer**To use Erlport, just add to your dependencies #mix.exs{:erlport, "~> 0.9"} Once Erlport is added to your project, you have access to module in your elixir code. This Erlport python module allows you to start an Elixir process to execute python code. :_python_ #creates and Elixir process for calling python functions{:ok, pid} = :python.start() #get the current python version:python.call(pid, :sys, :'version.__str__', []) Even better, Erlport allows you to configure python path so you canload custom python modules from a specific directory! #Automatically load all modules in directory with path /custom/modules/directory {:ok, pid} = :python.start([{:python_path, '/custom/modules/directory'}]) With the process returned from calling , you can call functions in your python module using the familiar - , , format start/1 MFA module function argument {:ok, pid} = :python.start([{:python_path, 'custom_modules_directory'}]) = :test #python module to call = :hello # function in module = ["World"] # list of arguments pass to function the functionresult = :python.call(p, module, function, arguments) module function arguments If the python function returns data, it will be bound to variable in the code above. Otherwise returns for python functions that don’t return any data. result :python.call/4 :undefined In this project, we will create a simple app that encode string to QRCode images. Then decode the QRCode image file to get the string back. Instead of using native Elixir/Erlang lib we will call Python module ( ) from our app to do the encoding and decoding. Check to setup qrtools. Let Create The Elixir Project qrtools here 1. Create the Elixir OTP app $ mix new elixir_python_qrcode 2. Add dependency Add to your dependencies erlport mix.exs defp deps do[# {:dep_from_hexpm, "~> 0.3.0"},# {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}, ,]end {:erlport, "~> 0.9"} Then install project dependencies. $ mix deps.get 3. Lets add wrapper Elixir module for Erlport Create , and add Elixir module called and add these helper functions. lib/helper.ex ElixirPythonQrcode.Helper defmodule ElixirPythonQrcode.Helper do@doc """ Parameters - path: directory to include in python path (charlist) """def python_instance(path) when is_list(path) do{:ok, pid} = :python.start([{:python_path, to_charlist(path)}])pidend def python_instance(_) do{:ok, pid} = :python.start()pidend @doc """Call python function using MFA format"""def call_python(pid, module, function, arguments \\ []) dopid|>:python.call(module, function, arguments) endend 4. Add Module code for the main. We will have two functions — which takes in some string and file path. It encodes the data and write the QRCode image to the given file path, and — which takes in file path to QRCode image and decodes to get original data. encode/2 decode/1 Let’s edit lib/elixir_python_qrcode.ex # lib/elixir_python_qrcode.ex defmodule ElixirPythonQrcode do@moduledoc """Documentation for ElixirPythonQrcode."""alias ElixirPythonQrcode.Helper def encode(data, file_path) docall_python(:qrcode, :encode, [data, file_path])end def decode(file_path) docall_python(:qrcode, :decode, [file_path])end defp default_instance() do#Load all modules in our priv/python directorypath = [:code.priv_dir(:elixir_python_qrcode), "python"]|> Path.join()Helper.python_instance(to_charlist(path))end wrapper function to call python functions using default python instance defp call_python(module, function, args \\ []) dodefault_instance()|> Helper.call_python(module, function, args)end end 5. Now let’s create our python module Create module that will contain our python functions. These functions handle the actual QRCode generation and decoding. priv/python/qrcode.py # priv/python/qrcode.py def decode(file_path):import qrtoolsqr = qrtools.QR()qr.decode(file_path)return qr.data def encode(data, file_path):import qrtoolsqr = qrtools.QR(data.encode("utf-8"))return qr.encode(filename=file_path) Let’s test our code. 6. $ iex -S mixiex(1)> ElixirPythonQrcode.encode("Some test to encode", "qrcode.png")0iex(2)> ElixirPythonQrcode.decode("qrcode.png")'Some test to encode'iex(3)> ElixirPythonQrcode.decode("qrcode.png") |> to_string()"Some test to encode" That’s it. Check out . the github repo Happy coding! Check out for Asynchronous communication between Elixir and Python Part II

Mixing Python with Elixir

With the recent release of CouchDB 2, the CouchDB ecosystem is on fire! There is now an out-of-the-box way of setting up a multi-master cluster of nodes that can scale to handle a ton of data. And, tools like PouchDB can allow your app to talk directly to CouchDB to enable your apps to be . It’s an exciting time to be using CouchDB at the data layer of your app! offline-first Even if you focus on a frontend-heavy design, chances are, you’re going to have to do a lot of coding on your backend to communicate with your CouchDB instance, including subscribing to real-time changes. When we first started working on a digital dropbox and grading system, we relied heavily on the awesome client. Through a lot of trial-and-error though, we found that a lot of messy logic, mostly to handle large data sets and transient database errors, was starting to clutter the application layer. We were also left wanting something flatter than nano that more closely mimicked the raw CouchDB RESTful API. We didn’t want to have to bind the client to a specific database, as in practice, this leads to a lot of overhead. We also wanted promises to be a native part of our client as we consider them essential to keeping our code manageable. Quizster, nano So, was born. Slouch Slouch provides all the that you’d expect to find in a CouchDB client and a lot more. Let’s take a look at some of the cool stuff. typical features Fault Tolerance Slouch attempts to retry your requests in the event that the connection to your database is temporarily lost. Moreover, it retries through max_dbs_open errors and other transient errors to prevent your application from crashing every time there is a hiccup on the database. StreamIterators These days, working with large data sets is practically a requirement. Sure, you can paginate your data, but CouchDB supports streams and StreamIterators make working with these streams a breeze. StreamIterators are available for all requests that can return a large number of docs, i.e. , , , db.changes() db.view() doc.all() system.updates() StreamIterators work with promises and reduce your code by introducing a simple and powerful pattern. Example 1 — Sequentially process all docs in a DB slouch.doc.all('mydb', { include_docs: true }).each(function (item) { // If we return a promise then the all() iterator won't move on  
// to the next item until the promise resolves. This allows us  
// to iterate through a large number of docs without consuming a  
// lot of memory. It also allows us to control the flow of the  
// docs and process them sequentially so that we don't end up  
// thrashing the processor with too many concurrent promises.  
return Promise.resolve('foo => ' + item.foo); }).then(function () { // Done iterating through all docs }).catch (function (err) { // An error occurred }); Example 2 — Process a max of 5 docs concurrently var Throttler = require('squadron').Throttler;var throttler = new Throttler(5); slouch.doc.all('mydb', { include_docs: true }).each(function (item) { return Promise.resolve('foo => ' + item.foo); }, throttler).then(function () { // Done iterating through all docs }); Helper functions The CouchDB conflict policy is a very powerful convention and is at the heart of CouchDB and its offline capabilities. Sometimes however, conflicts don’t matter to you and it’s nice to have functions to ensure that your logic persists through these conflicts. upsert Sometimes you just want to force a creation or update even if there is a conflict: slouch.doc.upsert('mydb', { _id: '1', foo: 'bar' }); getMergeUpsert getMergeUpsert allows you to make partial updates without regard to conflicts: // Create a docslouch.doc.create('mydb', { _id: '1', foo: 'bar' }).then(function (doc) { // Add the \`yar\` attr to the doc and ignore any conflicts  
return slouch.doc.getMergeUpsert('mydb',  
  { \_id: '1', yar: 'nar' }); }); Slouch and Beyond This just scratches the surface of what you can do with . We’ve found to work very well in production for and using it has greatly simplified our application layer as a lot of the boilerplate code is now at the layer. Slouch Slouch Quizster Slouch Our next step is to use to open source our layer so that it is a little easier to implement scaling replication and change listening for CouchDB. Slouch Spiegel About the Author Geoff Cox is the Co-Founder of a digital dropbox and grading system. uses a full stack of JS and runs CouchDB and PouchDB at the data layer. Quizster, Quizster,

Don’t Just Relax; Slouch: A JS Client for CouchDB that Does the Heavy Lifting

A classification approach with natural language processing in R Audience This short blog post is intended for someone who has basic understanding of the natural language processing algorithm in addition to familiarity with machine basics, particularly in classification techniques, and is keen to learn the actual integration of the two in R. We will go over detailed data preparation as well as model optimization. It is also recommended to review for a brief introduction on word2vec, learning this post word2vec. Objective The objective is to learn how the Dow Jones Industrial Average would be affected by daily popular news on . Reddit Dataset The datasets can be found . The simplified dataset is consisted of daily Dow Jones data and top daily headlines from Reddit WorldNews Channel for the past 8 years. The target variable is Label (0 is down and 1 is up). here Screenshot of the stock news data set Approach We will use the algorithm to first create word vectors of 100 dimensions on all the headlines. We will then transform the headlines into sentence vectors by averaging the word vectors that every sentence is consisted of. Finally we will use a variety of classification algorithms to predict whether Dow Jones Industrial Average would go up or down. word2vec Data Preparation 1…Remove undesired characters , including punctuations and upper cases 2…Remove English stop words such as “i,” “my,” “myself” Screenshot of the data set with unwanted features removed 3…Input cleaned headlines into model to obtain word vectors word2vec Screenshot of the word vectors generated from cleaned headlines 4…Apply and average word vectors to all headlines 5…Return 2-dimension vector representations on headlines using t-SNE 6…Create dataset with target variable and 2-dimension vectors Screenshot of the finished data set Model Evaluation Here we will use the package in R to evaluate the prediction accuracy on a variety of algorithms, including CART (rpart), logistic regression (glm), linear discriminate analysis (LDA), k-Nearest Neighbors (kNN), Support Vector Machine.with Radial Basis Function (svmRadial), and eXtreme Gradient Boosting (xgbLinear). caret It is observed that all algorithms above have unsatisfying performance, with close to 50% accuracy and poor Kappa values. However, SVM with the Radial Basis Function kernel tends to perform better than other algorithms, with a 95% confidence interval on accuracy between 51% and 55%. Performance on various classification algorithms As a result, we will explore a few optimization options. Performance Optimization 1…Stacking Multiple Models We will first try to stack all the models to see if performance will improve. Unfortunately, not much improvement is observed. 2…Averaging All Headlines Instead of treating top headlines as individual variables, we will combine all headlines to create one set of 2-dimension vector. Performance on the algorithms with averaged headlines A slight increase of 0.1% in mean accuracy is observed in SVM algorithm. 3…Shifting Target Variable We will try shifting the target variable back 1–7 days, as daily news may have lagged effects on . Alternatively, we can also shift the target variable forward a couple of days, however, this approach will contradict with the objective of stock marketing performance using daily top news. stock performance predicting Performance on the algorithms with 3 day lag on target variable We can observe that CART (rpart) now has the best performance in terms of mean accuracy at 53.66%. There is also another slight increase of 0.1% in mean accuracy is observed in the SVM algorithm. 4…Tuning Model Parameters Here we will demonstrate how to tune parameters of the CART and SVM algorithms to see if we can increase the accuracy on our predictions. CART In rpart, the tuning parameter is , or . We will examine the accuracy on different values of . The optimal cp value is found to be 0.15, as shown below. cp complexity parameter cp SVM In SVM with Radial Basis Function, the tuning parameters are and ( ). We will examine the accuracy on different values of and . The optimal values are shown below, with and . sigma C cost sigma C sigma = 0.05 C = 3 R Code library(devtools)library(tm)library(ggplot2)library(text2vec)library(rword2vec)library(wordVectors)library(tidyverse)library(Rtsne)library(caret)library(mlbench)library(caretEnsemble) setwd("~/Desktop/stock")#Simple Visualization on DJIA datastock <- read.csv("DJIA_table.csv")stock$Date <- as.Date(stock$Date) ggplot(data=stock,aes(x=Date,y=Open))+geom_line()+labs(title="Dow Jones Industrial Average") #Read the simplified data setdata <- read.csv("stock.csv") #Remove unwanted columns for natural language processingtext <- data text$Date <- NULLtext$Label <- NULL #Remove unwanted characters/punctuations/upper-casesfor (i in 1:25){text[,i] <- gsub('b"',"",text[,i])text[,i] <- gsub("b'","",text[,i])text[,i] <- gsub("[[:punct:]]", "", text[,i])text[,i] <- tolower(text[,i])} write.table(text,"text.txt") #Rename column namesnew_text <- subset(text,select=paste("Top",1,sep=""))new_text <- setNames(new_text,"x") for (i in 2:25){new_text2 <- subset(text,select=paste("Top",i,sep=""))new_text2 <- setNames(new_text2,"x")new_text <- rbind(new_text,new_text2)} write.table(new_text,"text.txt") final_text <- new_text #Remove stopwordsstopWords <- stopwords("en")'%nin%' <- Negate('%in%') train <- lapply(final_text$x, function(x) {t <- unlist(strsplit(x, " "))t[t %nin% stopWords]}) train.df = as.data.frame(do.call(rbind, train))train.df$x <- do.call(paste, c(train.df[,1:45], sep=" ")) finaltext <- subset(train.df,select="x") write.table(finaltext,"text_data.txt") #Input words to word2vecmodel = train_word2vec("text_data.txt", output="vec.bin",threads = 4, vectors = 100, window = 12, min_count = 5, iter=10, force=TRUE) #Plot the word vectorsplot(model)#Convert binary to text formatbin_to_txt("vec.bin","model1text.txt") #Remove first rows manually in the .txt file #Convert .txt to .cvsm1 <- read.table("model1text.txt",header = F,quote = "", row.names = NULL, stringsAsFactors = FALSE) colnames(m1)[1] <- "word" #Apply word vectors to headlines and return 2-dimension representations with t-SNElabel <- subset(data,select="Label")labeltext <- cbind(label,text)labeltext$id <- 1:nrow(labeltext)labeltext$id <- as.factor(labeltext$id) for (i in 2:24){s <- strsplit(labeltext[,i], split= " ")new <- data.frame(V1 = rep(labeltext$id, sapply(s, length)), V2 = unlist(s))colnames(new) <- c("id","word")new <- merge(new,m1,by="word")new2 <- aggregate(V2 ~ id, new, mean)for (j in 4:102){new3 <- aggregate(new[,j] ~ id, new, mean)new2 <- merge (new2,new3,by="id")}colnames(new2)[2:101] <- paste("V", 1:100, sep="")tsne <- Rtsne(new2, dims = 2, perplexity=100, verbose=TRUE, max_iter = 500)t = as.data.frame(tsne$Y)colnames(t) <- c(paste("top",i,"x",sep=""),paste("top",i,"y",sep=""))labeltext <- cbind(labeltext,t)} write.csv(labeltext,"labeltext.csv") #Add back Label informationmydata <- labeltextmydata[,2:27] <- NULL mydata$Label[mydata$Label 0]<- "No" 1]<- "Yes"mydata$Label[mydata$Label #Classfication Modelscontrol <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(Label~., data=mydata, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #1: Stacking Multiple Models-----#Model CorrelationsmodelCor(results)splom(results)#Model Stacking with GLMstackControl <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)set.seed(2)stack.glm <- caretStack(models, method="glm", metric="Accuracy", trControl=stackControl)print(stack.glm) #-----Performance Optimization #2: Averaging All Headlines-----mydata2 <- mydatamydata2$headline <- rowMeans(mydata2[,2:45], na.rm = TRUE)mydata2[,2:45] <- NULL control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(Label~., data=mydata2, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #3: Shifting Target Variable-----shift <-subset(mydata,select="Label")shift1 <- as.data.frame(shift[-1,])shift2 <- as.data.frame(shift[-c(1,2),])shift3 <- as.data.frame(shift[-c(1,2,3),])shift4 <- as.data.frame(shift[-c(1,2,3,4),])shift5 <- as.data.frame(shift[-c(1,2,3,4,5),])shift6 <- as.data.frame(shift[-c(1,2,3,4,5,6),])shift7 <- as.data.frame(shift[-c(1,2,3,4,5,6,7),]) mydata1 <- mydata[-1989,]mydata2 <- mydata[-c(1988,1989),]mydata3 <- mydata[-c(1987,1988,1989),]mydata4 <- mydata[-c(1986,1987,1988,1989),]mydata5 <- mydata[-c(1985,1986,1987,1988,1989),]mydata6 <- mydata[-c(1984,1985,1986,1987,1988,1989),]mydata7 <- mydata[-c(1983,1984,1985,1986,1987,1988,1989),] mydata1$Label <- NULLmydata2$Label <- NULLmydata3$Label <- NULLmydata4$Label <- NULLmydata5$Label <- NULLmydata6$Label <- NULLmydata7$Label <- NULL mydata1 <- cbind(shift1,mydata1)colnames(mydata1)[1] <- "label1day"mydata2 <- cbind(shift2,mydata2)colnames(mydata2)[1] <- "label2day"mydata3 <- cbind(shift3,mydata3)colnames(mydata3)[1] <- "label3day"mydata4 <- cbind(shift4,mydata4)colnames(mydata4)[1] <- "label4day"mydata5 <- cbind(shift5,mydata5)colnames(mydata5)[1] <- "label5day"mydata6 <- cbind(shift6,mydata6)colnames(mydata6)[1] <- "label6day"mydata7 <- cbind(shift7,mydata7)colnames(mydata7)[1] <- "label7day" control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE)algorithmList <- c('rpart','glm','lda','knn','svmRadial','xgbLinear')set.seed(1)models <- caretList(label3day~., data=mydata3, trControl=control, methodList=algorithmList)results <- resamples(models)summary(results)dotplot(results) #-----Performance Optimization #4: Tuning Model Parameters-----set.seed(3)control <- trainControl(method="repeatedcv", number=10, repeats=3, savePredictions=TRUE, classProbs=TRUE) #Tuning cp in CARTrpartGrid <-  expand.grid(cp = c(0.01,0.05,0.1,0.15)) rpartFit <- train(label3day ~ .,data = mydata3,method = "rpart",tuneLength = 10,tuneGrid=rpartGrid) rpartFitggplot(rpartFit) #Tuning sigma and C in SVMsvmGrid <-  expand.grid(sigma = c(0.01,0.05,0.1,0.15),C = c(1,3,5,7,9)) svmFit <- train(label3day ~ ., data = mydata3,method = "svmRadial",trControl = control,verbose = FALSE,tuneGrid = svmGrid) svmFitggplot(svmFit) Questions, comments, or concerns?jchen6912@gmail.com

Predict Stock Market with Daily Top News

Recently, I read a post on Hackernoon by Aashay Sanghvi titled “The Startup Scene at Harvard”. It was very well-crafted and I found it super interesting to take a peek into the startup scene of a top entrepreneurship school in the nation. It inspired me to write my own version of the post to give prospective Brunonians an idea of what Brown is like and to give past alums some pride on how much the scene has grown since they graduated. If you haven’t read his post yet, check it out below! _Dropping out is normally what comes to mind when people think of “Harvard” and “startups.”_hackernoon.com The Startup Scene at Harvard When you think of hubs for undergraduate entrepreneurs, is usually not the first university that comes to mind. Harvard, MIT, or Stanford usually take that laurel but Brown University is certainly on the rise. With the creation of a new centralized physical location for entrepreneurship, Brown’s scene is booming. Here’s the scoop on some of the things happening on Brown Street nowadays (as of Feb. 2017). Brown startup Clubs: EP is a student-run entrepreneurial initiative on campus and the biggest hub for young entrepreneurs. It is the umbrella organization for many great entrepreneurial resources on campus including Innovation Dojo, Startup Office Hours, Startup @Brown, and many others! Its alumni board consists of notable alumni such as Jeff Stibel, co-founder of Bryant Stibel, and Bob Goodman, a board member at Bessemer Venture Partners. This semester, its budget nearly doubled due to generous alumni funding so expect bigger and better things from them! Brown Entrepreneurial Program (EP) : Innovation Dojo is an EP workshop series that aims to introduce 1st and 2nd year Brunonians to entrepreneurship and the resources available to them on the hill. They hold meetings every week where experts give masterclass workshops on a variety of topics in entrepreneurship. All of this ties into a personal project where you work on your own startup or plan one out for the future and present it to the entrepreneurship community! Innovation Dojo : The Rhode Island School of Design (RISD) is the sister school of Brown and all clubs at both schools are open to students from both campuses. RISD E’Ship has a strong network of student entrepreneurs as well as its own startup accelerator program, E Partners Program, that provides funding and mentorship to student ventures. RISD E’Ship : HCRI is a student group dedicated to the advancement of robotics as a means to benefit human society economically and socially. Their aim is to create and understand robots that coexist harmoniously with humans. They often put up visual and interactive presentations on the bottom floor of the Computer Science building (CIT) to showcase their recent works. Humanity-Centered Robotics Initiative : A collaboration between the Warren Alpert Medical School of Brown University and the Rhode Island School of Design, Design+Health is an incubator which encourages students across all disciplines to develop and implement design solutions that address clinical and public health concerns. Design+Health : VC Presence: DRF is a student-focused venture fund of First Round Capital. They’re the largest student-run venture firm in the nation, funding and supporting an incredibly talented community of student entrepreneurs which now includes the Brown community, as they recently brought on Haris Memon (Brown ’19) as Head of Marketing for the firm. Dorm Room Fund : To date, Dorm Room Fund has made 125+ investments. Their portfolio companies have gone on to raise $200M+ in follow-on funding from investors like Andreessen Horowitz, Sequoia Capital, and Y Combinator. RDV, like Dorm Room Fund, is a student-focused VC firm backed by General Catalyst. It currently has a strong presence at universities in Boston and NYC, and our very own Valentin Perez (Brown ’18 + Former Co-President of Brown EP) is a partner. Rough Draft Ventures : Rough Draft Ventures has made 90+ investments, with portfolio companies raising follow-on funding of $230M+ from investors like SV Angel, Y Combinator, and Lerer Ventures. : Evertrue Ventures is a community of Brown alumni, investors, entrepreneurs, and innovators with a passion for supporting Brown alumni-led startups. Evertrue Ventures Events: Startup@Brown is a two-day conference hosted by EP every fall for startups across the nation come together on Brown’s campus. Many come to find new talent to join their team as well as hold office hours for young entrepreneurs looking for advice or someone to bounce ideas off of. Workshops led by industry professionals are held throughout the day where students can learn about accelerating their startup, plotting a career in tech, or building a healthy startup culture. This year, Startup@Brown was held in October with speakers coming from notable companies including Rough Draft Ventures, WayUp, and Techstars. Startup@Brown: We@Brown, the Women’s Empowerment Conference, aims to empower women to pursue their professional passions. Hosted by the Lady Launchers team at Brown EP, this year’s event will feature prominent female leaders in business and tech including Melanie Whelan, the CEO of Soulcycle, Laura Thompson, a VC Investor at Google Ventures and Co-founder of Runa tea, and Marian Salzman, the CEO of Havas PR North America. WE@Brown : In an all-expenses-paid trip, students will have the opportunity to visit a startup hub with their peers at EP. Activities include private tours of notable companies, dinners with successful Brown alumni, and opportunities to network with professionals in entrepreneurship and technology. In the past, Synapse has paid visits to NYC, SF, and LA, visiting companies such as KPCB, Apple, and Dropbox. In the future, Brunonians may have the opportunity to experience business cultures in a variety of different countries. Synapse: Hack@Brown is a two-day hackathon hosted by Brown, bringing hackers of all skill levels from all around the country together to spend a weekend programming, networking, and learning new technologies. Workshops are held before and throughout the event. Hack@Brown is a staple event for any CS concentrator at Brown. Hack@Brown: Nelson Center for Entrepreneurship A recent addition to the Brown startup landscape after a $25M donation from Jonathan Nelson ’77, the Nelson Center for Entrepreneurship acts as a nexus for entrepreneurial education by giving the community a visible space for all of its resources. Startup Office Hours, which brings highly-accomplished individuals from around the community to Brown to have one-on-one meetings with aspiring students entrepreneurs, calls the Nelson Center its home. The center also has five Peer Entrepreneurs-in-Residence (PIERS) who provide guidance and mentorship to those working on a new venture or are interested in learning how to join the entrepreneurial community at Brown. Along with these personal mentorship sessions, the center has held startup event series, , and film screenings for the community. The center is currently operating out of a temporary location right now but once the official center is completed next semester, its activities will only ramp up. major academic conferences Cool Startups at Brown: DocTalk mobile app that seeks to tackle many of the inefficiencies of the healthcare system in India. It facilitates interaction with your doctor by creating a clear channel of communication with them through the app, hosting on-the-go storage for all your medical files, and providing a means for effortless mobile requests for your prescriptions. Founded by Krishna Chaitanya Aluru ’17 of Brown, Akshat Goenka ’14 of Wharton, and Vamsee Chamakura ’17 of IIIT, DocTalk is currently going through Y-Combinator’s winter batch. DocTalk : is a Nomnomz offers two services: Nomnomz Checkout, which provides self-service checkout kiosks for restaurants, and Nomnomz Collegiate, a food pre-ordering service tailored for college campuses. It’s founder and CEO, ’18, is a strong contributor to Brown’s startup scene. Nomnomz : Matthew DiMarcantonio Bonsai is a high-end fashion and lifestyle marketplace HQ’d in Toronto. The team has raised over $780,000 in its first two funding rounds and is partnered with notable brands such as Acne Studios, Helmut Lang, and Balmain. Its Co-Founder & CFO, Kunal Khemani ’19, is currently taking a semester off in Toronto to devote full-time attention towards the platform. Bonsai : Speechify is a text-to-speech software that allows the user to listen to any text with a voice and speed of their choosing. It boasts multilingual support across accents and languages and makes efficient use of keyboard shortcuts for a smooth experience. Another nice feature of the app is the ability to create custom audiobooks instantly by highlighting any text on your phone or computer. Its founder Cliff Weitzman ’18 was recently recognized as a part of the 2017 Forbes 30 under 30 list in Education. Speechify: Bruno Innovation for Life: Our strong entrepreneurial spirit on campus has led to some amazing startups being founded by alumni, including: Casper ( ‘12, ’12, Gabriel Flateman ‘12) Neil Parikh T. Luke Sherwin Warby Parker (Andy Hunt ‘04) TeeSpring ( ‘11, ‘11) Walker Williams Evan Stites-Clayton Slideshare. ( ’98, Jonathan Boutelle ‘97) Rashmi Sinha Despite some of our alums raising over $100M for their startups and growing their teams in offices around the world, they still contribute heavily to the Brown community by offering advice and mentorship to current students and participating in conferences and workshops. If you’re interested in learning more about entrepreneurship@Brown, check out this recently released web hub, courtesy of Wesley Dencker ‘19 and the rest of the Tech team over at EP: _Entrepreneurship and Innovation at Brown and RISD_innovateonthehill.com Innovate on the Hill

he Startup Scene at Brown

There is a growing excitement in the developer community around the serverless paradigm, an idea that lets developers to deliver workloads without provisioning or managing servers. At its core, serverless architecture relies on orchestrating a vibrant eco-system of fully managed services. With increasing number of such backend services — namely backend-as-a-service (BaaS) — supported by all the key cloud infrastructure (IaaS) providers, I think it’s fair to say 2018 will be the year of serverless. With all the excitement, there’s also confusion as to what constitutes to serverless architecture. Conventional wisdom — catalyzed by cloud providers’ rhetoric — is that only Function-as-a-Service (FaaS) implementations represent a serverless architecture. I wrote this post to challenge some of those viewpoints in a hope to present a case for expanding this definition beyond functional level abstraction to application level abstraction to be considered as serverless. Serverless 1.0 The idea of an absence of the server concept during software development — or serverless — is not new. Heroku was perhaps the first to introduce this concept around 2008 that had broad adoption by the developer community (a case can be made about Google App Engine being the first, but will keep that for another post). With a developer first mentality, Heroku promised three main things: Cheaper, by isolating workloads in much small container images compared to VMs by IaaS providers, optimal to run most application workloads Comprehensive set of fully managed backends, accessible from a marketplace with federated identity and consolidated billing Great developer experience using a fantastic command line tool to develop, debug and deliver workloads For applications written in a language/framework that Heroku supports and adhere to a simple standard — called the 12-factor methodology — developers could follow a very familiar git-based workflow to deploy their code. Heroku would seamlessly handle the availability and scaling of the workloads. The declarative formats of 12-factor methodology promised to maximize iteration speed and provided a clean contract with the underlying OS, offering maximum portability As containers have much smaller footprint than VMs, they allowed Heroku to efficiently schedule workloads across a diverse set of machines with various configurations and optimize for price-performance. To make this a reality, Heroku built container runtimes and orchestration engines that ran on top of Amazon Web Services that were closed off from the user. The simplicity promised by the 12-factor design, however, lacked the flexibility do deploy workloads with complex needs. You couldn’t deploy any workload that was wasn’t 12-factor or have your workloads run in a private network. The marketplace — being comprised mostly of external vendors — paid little or no attention to latency between the application and backends that often resulted in degraded performance. The closed approached, the lack of flexibility and a degraded performance kept Heroku away from being adopted by enterprises with complex needs and drove it to remain as a much-loved platform for “hobby” developers. This gap in the market gave birth to platforms — called platform-as-as-services (PaaS) — like CloudFoundry (an EMC and VMWare spin-off), Makara (RedHat OpenShift) and DotCloud (Later renamed to Docker) By embracing an open approach, these platforms promised the ease-of-use of Heroku style workflows while providing the flexibility or using your own hardware with a focus on simplicity Like Heroku, these platforms also relied heavily upon Linux containers — a relatively old but unknown concept at the time — as a packaging and isolation mechanism to deliver workloads. The native container runtime provided by Linux — called LXC — was hard and unpleasant to use; limited its adoption to advanced teams at companies like Google and ones building infrastructure tools. DotCloud, a relatively unknown company with a product that couldn’t differentiate from the competition — facing a possibility of extinction — open sourced a tool they’ve developed to simplify the use to LXC called “Docker.” Docker’s key to simplicity was the image format — contents the image, declared in a manifest file is then used to build an image that can be built anywhere and shared with others by storing in a central repository. This build once, run anywhere approach gave way for developers to solve the particularly tricky, development-to-production parity problem. It didn’t take long for developers to embrace Docker that lead to a Cambrian exposition of the cloud-native ecosystem. While Docker provided the image format and runtime for a single workload, tools like Kubernetes were born to orchestrate a variety of these workloads and schedule them in a fault tolerant manner. What began as a serverless promise by Heroku — the application runtime abstraction — paved the way for a thriving ecosystem that changed the fundamental way we design the delivery of distributed workloads Serverless 2.0 With increased number of workloads moving to the cloud, it didn’t take long for IaaS providers to understand the value of providing fully-managed backends — such as databases, caches, load balancers, etc — the support infrastructure required by the workloads For providers such as Amazon Web Services (AWS), that began with a simple, managed object store offering, expanded into over 70 services (as of the writing of the post); for many providers, a way to orchestrate these rich set backends was evident next step. By leveraging the orchestration and containerization wave, it made it possible to rethink abstraction levels. This gave birth to a new level of abstraction — function-as-a-service. AWS Lambda is a product that provides functional abstraction with the same promise of faster iteration speed, zero administration, and a much granular pricing model. The difference here is that development focus is on a single function or module rather than a service with a large surface area like in the application runtime abstraction. Another fundamental difference between application level abstraction and functional level abstraction is where and how you define invocation mechanism for workloads. In application abstraction, the invocation mechanism is described in the application code itself and provides the flexibility for the developer to choose the framework. For functional, the invocation is defined in a manifest file designed by provider. In case of AWS Lamda, an asynchronous invocation by placing a message on a managed queue (SQS), and a web-based invocation by calling a predefined endpoint on a managed router (API Gateway) In essence, FaaS is an application framework created and managed by the provider. This makes FaaS appealing for developers that don’t necessarily care about vendor lock-in and are flexible to adopt the provider’s manifest in exchange for a piece of mind. Conclusion It’s evident by now that application and functional runtime abstraction deliver the promise of serverless — a future where developers can focus on writing code that provides functionality to their users instead of dealing with complexities of delivering that code.

Serverless is bigger than FaaS

I love and I love the productivity and flexibility that comes from a good command line interface. If you have read any of my previous articles here: JavaScript _Read writing from Mike Parsons in Hacker Noon. Software Research Geek. Every day, Mike Parsons and thousands of other…_hackernoon.com Mike Parsons - Hacker Noon you know that I am a big fan of getting a lot done in a few lines of code. I spend most of my time inside the browser console and while I love the power and expressiveness of the development tools, they are often overkill for what I want. I also like having a consistent JavaScript Console interface that is available from anywhere, not just inside the browser. For example, there are times when I want to use JavaScript outside the browser to automate Window’s Office Applications or script or use the browser in conjunction with native applications like SQL Server to perform some data tasks that are just not available inside the browser sandbox. One common use case I have frequently is to scrape some data from a web page and insert it into a native application like Microsoft Excel for further data analysis. Node.js While each of the environments listed in the title have there own console environments and REPL’s, I have found the command line interfaces limited in terms of integration with the host Operating System. Most operate inside a sandbox that makes perfect sense for the casual user but really restricts the capabilities I want as a power user or system’s engineer. Another use for this console is during presentations or live coding sessions when you want to demonstrate ideas quickly. I use it frequently to prototype ideas and to validate features available in the browser or the underlying OS. My Universal JavaScript Console is based on the terrific work done by Remy Sharp _jsconsole - Web based console - for presentations and workshops_github.com remy/jsconsole I took Remy’s excellent console and extended it with a few features that allow me to host the console in multiple environments. Let’s get started with the browser interface, found here: _Universal JavaScript Console_s3.amazonaws.com Console When you first navigate to the console, you are presented with this interface: You can start entering any valid JavaScript commands and immediately see the output. Try entering a few commands: Date()locationnavigator.userAgent1+1 You can enter any valid JavaScript, including setting variables and/or creating new functions. Many times you want to leverage an existing JavaScript library such as jQuery, lodash or the amazing ALASQL that allows you to use SQL in the browser. You can simply load the library via it’s URL, for example: I have added a few shortcuts for libraries that I frequently use: jqueryprototypedojomootoolsunderscorelodashmomentyuialasql In addition to loading JavaScript Libraries, you can also load HTML content from another Web Site. This is useful in scraping web pages, for example: These previous examples demonstrate using the Console in standalone mode. This is useful but my number one use for the Console is by leveraging it in the context of an existing Web Page or Site. In this mode, you basically inject the Console inside an existing Web Page and it acquires the context of the page so you can script the contents of the page just as you would if you opened the browser development tools. However, unlike the browser console, which is sandboxed via the web site settings, the Universal Console provides a great deal more power in terms of bringing in existing libraries into an existing web page and/or via extended network capabilities, like WebSockets or cross domain XMHttpRequest calls. This is accomplished via the use of a Bookmarklet, which you can access via the command. Simple drag the Bookmarklet from the help page to your browser Bookmarks and it becomes available from any web site. :help To illustrate this functionality, we are going to navigate to and into this page so we can script it’s contents. For this example, we are going to use the Console and one of my favourite Libraries (ALASQL) to get a list of all the new’s links on the HackerNews web site and export those to Excel. We are going to do this with 2 lines of code: https://news.ycombinator.com/newest Inject the Console :load alasql alasql("select href,innerText INTO XLSX('news.xlsx',{headers:true}) from ? where className='storylink'",[[...page.document.all]]) That’s pretty cool! When you inject the console into the page, it loads an external page ( ) and creates a new variable within the console. This variable is a reference to the JavaScript window object of the host page, so you can access any properties or methods of that page (i.e. , etc, etc). make sure you allow pop ups! page page.document, page.location Unlike the sandbox settings of the host page, however, the console has access to any web page, JavaScript library or Network endpoint (via WebSockets or XMLHttpRequest). Hopefully, this gives you some idea of the power of the Universal Console. Electron Hosting In the title of this article, I mentioned that the Console is also available in environments other than just the browser. First, lets look at how we host it in Electron. The first thing you need to do is download Electron via _Build cross platform desktop apps with JavaScript, HTML, and CSS._electron.atom.io Electron Once you have Electron installed and running, you need to create a new Electron Application, which in this case is a simple as creating a folder(call it console or whatever you like) on your machine and adding these 3 files to the folder: package.json {"name": "console","version":"1.0.0","main": "index.js"} index.js const {app, BrowserWindow} = require('electron')const path = require('path')const url = require('url') let win function createWindow () {win = new BrowserWindow({width: 800, height: 600})win.loadURL(url.format({pathname: path.join(__dirname, 'console.html'),protocol: 'file:',slashes: true})) win.on('closed', () => {win = null})} app.on('ready', createWindow) app.on('window-all-closed', () => {if (process.platform !== 'darwin') {app.quit()}}) app.on('activate', () => {if (win === null) {createWindow()}}) console.html <html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title></head><body></body><script src=" "></script></html> https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js Once you have created these files, simply drag the folder on the electron executable to run it: You should see something like the following: And then you can start typing any commands that reference the Electron, Chrome Browser or Node.js objects: NW.js Hosting Hosting in NW.js is just as easy. Download the application from here: _nwjs_nwjs.io NW.js As in the Electron example, create a local folder and add these two files: package.json {"name": "console","version":"1.0.0","main": "console.html"} console.html <html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title></head><body></body><script src=" "></script></html> https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js Again, just drag the folder onto the executable to run it and start typing commands to access the NW.JS environment: nw Note that both NW.JS and Electron host Node.js so you can use the Universal Console to script against the Node.js context in both these applications: Finally, we come to my most frequent usage of the Universal Console … Hosting the Console in a Window’s Application. You might say that both NW.JS and Electron are Window’s Applications and you would be correct. However, there are a lost of scenarios where Electron, NW.js as well as Node.js fall short in terms of native Window’s functionality, in particular, COM Automation and specifically Microsoft Office integration. Most businesses still run Microsoft Excel, Word, Outlook, etc and I wanted a way to script these applications using my favourite language, JavaScript. While the Window’s Developer Ecosystem has a ton of ways to build and extend Office Applications, they usually require a ton of dependencies and skills in using C# or Visual Studio, which in my case is overkill if I just want to do something quick and easy. So, we are going to host the Universal Console in a Window’s Application using a technology that already exists on every Window’s machine — the Window’s Scripting Host, and in particular, the HTML Application, or HTA: _On the other hand, an HTA runs as a fully trusted application and therefore has more privileges than a normal HTML file…_en.wikipedia.org HTML Application - Wikipedia This is as simple as creating a text file called console.hta on your local machine: console.hta <html><meta http-equiv="X-UA-Compatible" content="IE=Edge"><head><title>Console</title><HTA:APPLICATION ID="oHTA"APPLICATIONNAME="Console"BORDERSTYLE="normal"CAPTION="yes"CONTEXTMENU="no"ICON="favicon.ico"INNERBORDER="no"MAXIMIZEBUTTON="yes"MINIMIZEBUTTON="yes"NAVIGABLE="yes"SELECTION="yes"SHOWINTASKBAR="yes"SCROLL="no"SCROLLFLAT="yes"SINGLEINSTANCE="yes"SYSMENU="yes"VERSION="1.0"WINDOWSTATE="normal"/></head><body></body><script src=" "></script></html> https://s3.amazonaws.com/mparsons/jsconsole/js/inject.js That’s it! Save this file on your local machine and double click to run it: It works the same as in other environments with the important distinction that it now has access to the entire Window’s Ecosystem via COM integration. That means, you can do stuff like talk to Excel, Word, script Internet Explorer to scrape web sites or use third party applications that have a COM (or OLE Automation Style Interface), like Adobe, AutoDesk, Microsoft Dynamics, etc. Indeed, you will find that most Window’s Applications can be accessed and programmed this way. In this simple example, I am going to load Excel and read and write some simple values using the Excel Object Model. As a security measure, Window’s ask for permission to allow this: This show’s how easy it is to script Excel: Finally, in this next example, I am going to load the contents from an external Web Site and paste the values into the spreadsheet: So the following 6 lines of code are all that’s needed to perform this task: //Load our data from the web :load https://news.ycombinator.com/newest //Assign it to a variable for easy reference [Optional] var doc=docs[" https://news.ycombinator.com/newest "] //load our web page scaping library :load alasql //Run our query to scrape the data var data=alasql("select href,innerText INTO TAB({headers:true}) from ? where className='storylink'",[[].slice.call(doc.all)]) //copy it to the clipboard clipboardData.setData ("Text", data); //Paste it into Excel wb.ActiveSheet.Range("A1").PasteSpecial() Hopefully, this demonstrates the power of our Universal Console. You can access an early version here and I hope to document and share the code on Github soon. _By Mike Parsons_s3.amazonaws.com Universal Console If you like this article, please give it a recommendation! Thank you!

Universal JavaScript Console for Browser, Electron, NW.js, Node.js and Windows

(Edit) I have added some gif to this article. I’m glad you like ! In fact, I am a die hard fan and I’ve been using it for ages (since ST2). I have a few tricks I would like to share them with you. Sublime Text 3 This is not a language oriented article. This article ranges from use of ST3 and we are going to learn and do: basics to advanced Fuzzy searching basics. What the files under folder are. User Some important functionalities as a reminder. Working with words and lines. Selection and deleting techniques. Useful keys and advance techniques. navigation Project Management. Install a few handy plugins. Optimisation by changing a few settings. Add missing to the commands Command Palette. In case you don’t know how to open the Command Palette, hit . The is for files, otherwise known as . ctrl+shift+p ctrl+p go to go to anything Fuzzy searching It works like this: if you are after a term in any palettes, for instance , you don’t need to type in the whole thing. I would do which matches the term. Also there is no space needed. Preferences: Key Bindings kybi Files under User folder There are some under . I’m using GNU/Linux, but the principals are the same in other OSs. The contents should be in valid format, otherwise they don’t work. files ~/.config/sublime-text-3/Packages/User JSON The is for global settings. It can be accessed by command palette ( ). Preferences.sublime-setting Preferences: Settings The is where you can add key shortcuts. It can be accessed in command palette ( ). Default (Linux).sublime-keymap Preferences: Key Bindings The contains the mouse actions. For now you only can open it by browsing to the directory and edit. However in this article I will show you how to add it to the Command Palette with ease. Default (Linux).sublime-mousemap There is another kind of file that doesn’t exist in that folder by default but you can create one, which the functionalities, I will explain it in below. extends Command Palette Basic functionalities and concepts can be invoked in any of these ways: Multi cursor Select multiple lines, then hit ctrl+shift+l on different places ctrl+left-click Find under expand. Put your cursor on a word, then hit it will select that word. Then hit a few more times to find similar words while creating a new cursor. It also works with partially selected words (a few characters or a few words/lines). ctrl+d ctrl+d Find all: and click on ctrl+f Find All shows the current file’s location in the sidebar and highlights it. Right click on the view and choose . We will create a command for it below as it doesn’t exists (yet). Reveal file in sidebar Reveal In Sidebar by hitting . You can create a command for this one too if you want. Toggle Sidebar ctrl+k ctrl+b is an important one. You can invoke its palette by . It shows you a list of available files in your current project (see below). It too obeys the fuzzy searching rules. Here are more things you can do with it: Go To Anything ctrl+p shows a list of symbols (classes/functions/methods) you can navigate to. @ takes you to a line by typing the line number. : shows you a list of token words (they can be set up in files and they usually come with plugins you install) # .tmLanguage If you don’t type in any file names, they work on the current file. Always have a mental image of your project and possible symbol names and navigate with this palette. Example 1: If you want to go to a file called and you know you want to land on a function called , type this in the palette: . This is just an example and the letters don’t have any significance other than matching. You need to make sure the destination file is focused then add to the mix. brew_beer.go Ingredients() bre@ingrd @ Example 2: If you want to go to the line of a file called , you can type in . 18 drunk_snake.py drnksnk:18 extremely useful for using in multiple panes (more about it in the origami section) New View into File while keeping the cursor at its position can be done by . Note that if you don’t have any selection, the cursor will travel by the edge of the view. In case you want to avoid and keep the cursor where it is, select one word and scroll. It retains the cursor position. Scrolling ctrl+up/down is handy when your lines are long and don’t fit into the view. You can invoke it with Command Palette. Wrap lines can be done with . It uses the ruler position to wrap. Wrapping a paragraph alt+q More useful actions will be mentioned in other sections of this article, please read on. Working with Words and Lines Let’s assume you have your cursor on a word/line and you want to: Go to next/previous word: ctrl+right/left Go to matching bracket: . It also works if you are not a bracket. It will navigate you to the closest bracket. ctrl+m Select the : hit . Please do not go to the end/beginning of the word and do , those combinations have a different purpose. It’s not what a would do! current word ctrl+d ctrl+shift+left/right ninja Select to next/previous word as you go: ctrl+shift+right/left Delete to the : end of a word ctrl+delete Delete to the : beginning of a word ctrl+backspace Delete to the : end of the line ctrl+shift+delete lines: Duplicate ctrl+shift+d Delete current line all together: ctrl+x up/down**:** Swap lines ctrl+shift+up/down . It expands the selection in blocks, and expands more if you hit it more times until it selects the whole file. Expand Selection: ctrl+shift+space Delete a block: hit until you covered the block you want to remove, then delete. ctrl+shit+space Indent line(s) left: and Indent line(s) right: ctrl+[ ctrl+] Jump forward and back (settings all the way down on this article): as you are moving around in your project, you can always go back and forth to the place you were. This is handy for inspecting the project. Insert a tab where ST3 insists on a completion: shift+tab (Edit 1) When you copy/cut stuff in ST3, it keeps them in a history. If you do it shows you a pop up of the history and you can choose to paste it. This is super useful! ctrl+k ctrl+v Join Lines: ctrl+j Project Management Projects Projects contain a series of files/folders added into a project. You can add multiple folders to a project if you wish. I have a folder in my folder called where I keep all my ST3 projects. I also put them into their own sub-folder to manage different workspaces. I suggest you do the same as it clutters the whole folder if you work with a lot of projects. You can open the project settings by invoking in command palette. Here are a few tips: Projects Sublime Text Project: Edit "folders": [{"file_exclude_patterns": ["*.test", "*.out"],"folder_exclude_patterns": ["tmp"],],"path": "/path/to/project/folder",}], Workspaces You can have multiple workspaces per project. They share the same settings as their parent project, therefore when you change the project settings it applies to all of its workspaces. Use them if you have different layouts or you have different concerns while working on a project. You can save them in the same folder you keep your ST3 project by choosing menu. You can also create a brand new workspace for the current project from the menu. Project> Save Workspace As.. Project Fast Switching Projects Every time you open/create projects or workspaces, ST3 keeps a history of what you’ve been working on. You can then hit to navigate between projects or workspaces easily. ctrl+alt+p Handy Plugins is useful for aligning cursors. I have changed the key binding to Alignment { "keys": ["super+a"], "command": "alignment" } Make a multi cursor and hit to align them. It will move the text with which is really handy. You can create such sight in your file: super+a shows you the opening and closing pairs of brackets. I use it to identify the block I am working on. Bracket Highlighter can change word/character cases. It has these modes: Case Conversion snake_case camelCase PascalCase dot.case dash-case Separate words Separate with forward slashes Separate with backslashes Toggle between snake_case, camelCase and PascalCase can change the file name. I use this to depend less on my mouse. File Rename shows git changes in the gutter. You also can view the diffs and roll them back, really handy. Git Gutter I use it every all the times. You can land your cursor on an indented area (maybe a bracket or beginning of a line) and hit to jump to the respective indention. Jump Along Indent: alt+up/down is a must have one if you do on github. Just generate a token on your github account (with only gist enabled permission) and add it to the plugin. You never need to browse to gists web page for your gists any more. You also can load a gist, edit and save it. It updates the gist automatically. HaoGist gists can move your tabs like in your browser. MoveTab is useful if you increment numbers. Create a bunch of cursors and type then select all those zeros and hit . It makes a series of incremented numbers. You can seed it with any numbers. Increment Selection 0 ctrl+alt+i is another must have plugin to make todo lists. It also tracks the time spent on each item. It is very straightforward and starts with its own tutorial when you install it, so there is no need to explain it here. PlainTasks will enhance your sidebar. Make sure you install this one. SideBarEnhancements is a better version of sublime’s bookmark system. You can name your bookmarks and navigate them in the command palette. Very handy. Sublime Bookmarks looks really cool, give it a go. Material Theme : this awesome plugin adds some cool functionalities to multi-cursor functionality. For instance you can edit individual cursors while retaining the who multi-cursor state. You also can cycle through the cursors and edit them! Head over to its page to see the full description. (Edit 1) MultiEditUtils : I mention this the last, because I also wanted to tell you a little story. Please read this part as I explain the decisions I have made to set this very plugin. Origami important I can’t live without . The way ST3 handles panes is way primitive in comparison to tmux and my ST3 work flow has been a bit slow before I started using origami. Before using origami, I rarely utilised panes and only relied on goto anything. The very first time I saw origami, it blew my mind, but soon I left it for the following reasons: tmux The shortcuts were different than my tmux set up It made me slower to code because it didn’t feel natural My tmux prefix is and I couldn’t use it for origami because it is also functionality. Until I asked myself these questions: ctrl+a select all How many times am I selecting the whole file’s contents in a day? Which one contributes to my productivity: Selecting all or having a more flexible workspace? The answer to the second question is obvious, duh! However the answer to the first question turned out very interesting: at most 10 times a day! Now the other obstacle: ? Here is what I came up with, which worked amazingly well for me and I’ve been a happier programmer. I changed the select all to: How am I going to set these two up in such a way I can have best of both { "keys": ["ctrl+a", "ctrl+a"], "command": "select_all" } And changed all prefixes of origami to . Now here is what happens: ctrl+b ctrl+a My pane control/navigation is like the tmux set up, and it feels very natural to me. exactly If I want to select all, I do twice, why not?! ctrl+a Problem resolved. I also disabled creating empty panes, which is really annoying. This means if you do a it creates an empty pane with nothing in it, which is not useful unless you want to create a new file. You can achieve that by creating a new file and move it to a new pane by . In origami’s user settings I have: ctrl+a right ctrl+a shift+right "create_new_pane_if_necessary": false, And in the preferences I added this to close empty panes when the last file is closed: "origami_auto_close_empty_panes": true Now here is a few handy tricks for origami. I call it here, the key I have set to which is by default. Lets call arrow keys a direction. For instance this combination means you hit then hit . prefix ctrl+a ctrl+b up/down/left/right prefix, ctrl+direction ctrl+a ctrl+up creates pane on the direction. prefix, ctrl+direction destroys pane on that direction. prefix, ctrl+shift+direction travels to the direction. prefix, direction travels while carries the file to the direction. prefix, shift+direction clones (makes a new view on the file) while bringing the new view to the direction. prefix, alt+direction zooms the pane. prefix, ctrl+z un-zooms the pane back to normal. prefix, ctrl+shift+z shows you an input box to put in the proportion of pane’s horizontal (column) size. I’ll explain in a bit. prefix, ctrl+c shows you an input box to put in the proportion of pane’s vertical (row) size. prefix, ctrl+r The sizes work like this: If you are on a pane that has only one pane of its side (or above/below): 0.3 means the the occupies 30 percent of the space in respect to the whole space. pane you are on If you are on the middle pane: 0.2, 0.9 means the left (or top) most pane gets 20 percent, the right one gets 90 percent and you get the rest. In the first point by whole space I mean if you have three panes and you are on the first one, it accounts the percentage as if there is only two, and it pushes the other panes to make room. : You can make more and put them side by side in order to work on different portion of the file and navigate through the panes as opposed to jumping up and down. A tip Views Into a File Origami has a concept of saving layouts. You also can create a layout and save the workspace. Either one works, but I prefer to save the layout. Now you can get rid of the sidebar and enjoy more space. Changing Some Settings I found these settings to be useful: {"bold_folder_labels": true,"close_windows_when_empty": true,"detect_slow_plugins": false,"enable_tab_scrolling": false,"ensure_newline_at_eof_on_save": true,"fade_fold_buttons": false,"font_face": "Inconsolata-g for Powerline","font_options":["subpixel_antialias","gray_antialias","directwrite"],"font_size": 11,"line_padding_bottom": 3,"line_padding_top": 3,"margin": 0,"caret_extra_bottom": 0,"caret_extra_top": 0,"caret_extra_width": 1,"caret_style": "wide","tab_size": 4,"spell_check": true,"dictionary": "Packages/Language - English/en_GB.dic","file_exclude_patterns":["*.sqlite3","*.exe","*.pyc","*.pyo","*.dia"],"folder_exclude_patterns":[".svn",".git",".gitignore",".hg","CVS","_build","dist","build","site"],"highlight_line": true,"highlight_modified_tabs": false,"indent_guide_options":["draw_active"],"match_brackets_angle": true,"material_theme_accent_orange": true,"material_theme_contrast_mode": true,"material_theme_small_statusbar": true,"material_theme_small_tab": true,"material_theme_tabs_autowidth": true,"origami_auto_close_empty_panes": true,"save_on_focus_lost": false,"shift_tab_unindent": false,"show_encoding": true,"show_line_endings": true,"show_tab_close_buttons": true,"theme": "Material-Theme-Darker.sublime-theme","translate_tabs_to_spaces": true,"trim_trailing_white_space_on_save": true,"use_tab_stops": true,"word_wrap": "false"} Notice the file_exclude_patterns and folder_exclude_patterns are also in project settings, when you set it here it applies it to every projects you have. I have excluded .git because I don’t want ST3 to search in those files when I do to search in everything. ctrl+shift+f Commands Create a file by any name and extension. Like so: and add this to the file: sublime-command arsham.sublime-command [{"caption": "Open Mouse Settings", "command": "open_file", "args": {"file": "${packages}/User/Default (Linux).sublime-mousemap"}},] Then invoke the Command Palette and type in . It will open your mouse map file (I have mentioned this earlier in this article). There is a wide range of commands you can do. In fact, I would like you to do an experiment. Open the by hitting then type this in there: Open Mouse Settings Console Panel ctrl+` sublime.log_commands(True) Without closing the console, do something to sublime. Like toggling the The short key is . You can see in the console it says: SideBar. ctrl+k ctrl+b command: toggle_side_bar Now you know the command name, you can add this to your commands: {"caption": "Toggle My sidebar please", "command": "toggle_side_bar"} And invoke it with command palette. Go ahead and try a few more! Here are some useful commands: {"caption": "Reveal File In SideBar", "command": "reveal_in_side_bar"},{"caption": "Open Mouse Settings", "command": "open_file", "args": {"file": "${packages}/User/Default (Linux).sublime-mousemap"}},{"caption": "Open Project", "command": "prompt_open_project_or_workspace"},{"caption": "Fold Comments", "command": "fold_comments"},{"caption": "Unfold Comments", "command": "unfold_comments"},{"caption": "Fold One", "command": "fold_by_level", "args": {"level": 1}},{"caption": "Fold Two", "command": "fold_by_level", "args": {"level": 2}},{"caption": "Fold Three", "command": "fold_by_level", "args": {"level": 3}},{"caption": "Fold Four", "command": "fold_by_level", "args": {"level": 4}},{"caption": "Fold Five", "command": "fold_by_level", "args": {"level": 5}},{"caption": "Fold Six", "command": "fold_by_level", "args": {"level": 6}},{"caption": "Fold Seven", "command": "fold_by_level", "args": {"level": 7}},{"caption": "Fold Eight", "command": "fold_by_level", "args": {"level": 8}},{"caption": "Fold Nine", "command": "fold_by_level", "args": {"level": 9}}, Here are some useful key bindings: { "keys": ["super+a"], "command": "alignment" },{ "keys": ["alt+left"], "command": "jump_back" },{ "keys": ["alt+right"], "command": "jump_forward" },{ "keys": ["ctrl+0"], "command": "reset_font_size" }, Please let me know in the comments if you have any suggestions how to improve this article, thanks! Conclusion Sublime Text 3 is really fast and comes with a lot of functionalities. When extended with plugins can be the best editor around. Please try to use your keyboard and rely less on mouse navigations. Enjoy!

Stories, News & Mentions about DOTCAL

Dotcal’s Stories on HackerNoon