In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the big data community. Whether it’s a promotion,

Big Data Career Notes: October 2023 Edition

While more than 60% of global data leaders said they already have Generative AI baked into their plans, 69% of data leaders in APAC said the same. While 29% of APAC data leaders said they are embedding AI into the DNA of their businesses, only 18% in Europe and 20% in the US said the same.

Dataiku regional VP explains why APAC is leading the AI charge

Dataiku’s AI tooling is helping UK Higher Education access charity The Brilliant Club better identify students at risk of failing - improving their chances of getting into university

How AI is helping disadvantaged UK young people get into top universities

Dataiku has appointed Sandrine Bossard as their Chief People Officer, highlighting their commitment to employee development.

Dataiku appoints Sandrine Bossard as Chief People Officer to drive forward-thinking HR initiatives

Dataiku, the platform for Everyday AI, today announced the appointment of Sandrine Bossard as their new Chief People Officer.

Dataiku Welcomes Sandrine Bossard as Chief People Officer

Paris, France, Oct. 18, 2023 (GLOBE NEWSWIRE) -- Dataiku, the platform for Everyday AI, today announced the appointment of Sandrine Bossard as their new Chief People Officer. This significant ...

(MENAFN- GlobeNewsWire - Nasdaq) Paris, France, Oct. 18, 2023 (GLOBE NEWSWIRE) -- Dataiku , the platform for Everyday AI, today announced the appointment of Sandrine Bossard as their new Chief ...

Dataiku Welcomes Sandrine Bossard As Chief People Officer

Dataiku

dataiku

Data science is the new hotness, with thousands of job postings (some of which really aren’t data science), to dozens of platforms promising to help professionals in the field do their job more effectively. In typical fashion, not all these tools are new, but re-purposed for new use cases, with tools such as Python, R, and Hadoop experiencing new surges in interest thanks to the ‘new’ field of Data Science. One of the most well conceived and cohesive tools I’ve seen is Dataiku. It aims to package together all the tools that a data scientist and the teams that work with them might need in one application. To experiment with you will need a decently sized dataset, I opted for the time-honored , but for the sanity of my laptop, used a couple of gigabytes of the data. Dataiku NYC Taxi trip records Dataiku consists of a handful of open-source components (many of which you might recognize), but the software is closed source bound together with proprietary code, that you can install locally or in the cloud. For this review, I will use the Mac version of the free desktop client. with free and enterprise editions Download the application, run it, and your browser will automatically open to Then head over to the section and choose one of the helpers to get you started, I chose the ‘Tutorial 101 Starting project.’ http://localhost:11200. New project You can import data from a local or server file system, Hadoop, a variety of SQL and NoSQL sources, cloud storage providers, and further options provided by . After scanning your data, Dataiku provides a preview and some options for tweaking the import and schema, then you’re ready to create your dataset by clicking the green button. plugins create Next, you will see the screen where you can view, filter, sort, and analyze (provides a column based overview) your data. There are also processors for certain data types, for example, geocoding location data. You can create a wide variety of charts by dragging and dropping fields, or switching between types for a preview. Data exploration Useful so far, but you can also mix and match the GUI interface with Python, R, and SQL, if you have ever used Jupyter notebooks, then the style will be familiar to you. I’m no Python programmer, but thankfully there’s also a built-in console and debugger to help me figure out what the problem is. For the non-coders, Dataiku offers built-in machine learning models for prediction and clustering of data, and the ability to create your own learning models and train them. Again, creating your own is a matter of clicking, dragging and selecting options, for example, I created a model to show me what taxi pickups fell on weekends and public holidays in the US. And finally, to assemble all these components together is the workflow section where you can define which steps to run, and in what order, triggered manually, or programmatically via a REST API. This scratches the surface of what anyone needing to process and analyze large data sets can accomplish with Dataiku, and you can find more details on their , or listen to the interview I conducted with Claude Perdigou, a product manager with the company. website https://soundcloud.com/gregarious-mammal/dataiku-interview Originally published at dzone.com .

Data Science in a Box With Dataiku

🚀🤖✨leveraging Next-Gen RAG Digital Assistants for Enterprise Success 

Artificial intelligence (AI) has revolutionized the way we live, work, and interact with the world around us. From to , AI is quickly becoming an integral part of our daily lives. And it’s going to become more prevalent with each passing day, as the tech will be "as transformative as the Industrial Revolution," if not more so. self-driving cars virtual assistants some believe While previous decades were defined as an (where little AI research, development, or funding occurred), much has changed recently – especially with the release of generative AI tools like and . That has officially thawed, with massive influxes of pouring into both startups and Big Tech giants alike. With all that money and growth, the demand for skilled AI professionals has never been higher. AI winter ChatGPT Midjourney winter chill investments So, whether you're a seasoned professional or just starting out, there's never been a better time to work in the AI revolution. In this blog post, explore the best AI companies to work for in 2023 and the exciting career opportunities they offer. 1. OpenAI is a research organization dedicated to advancing artificial intelligence in a safe and beneficial manner. The organization was founded in 2015 by a group of tech industry leaders, including Elon Musk and Sam Altman. OpenAI aims to create AI systems capable of understanding natural language, learning from experience, and achieving human-like intelligence. OpenAI Recently, OpenAI has been making headlines for its innovative language model, ChatGPT, which can generate human-like responses to text prompts. This breakthrough has sparked excitement among researchers, developers, and businesses who can use ChatGPT to improve customer service, generate content, and automate tasks. It has also set the record for the fastest-growing consumer application in history, with over 100 million monthly active users in just two months. With such monumental growth, it should be no surprise that the company is actively recruiting talented individuals passionate about AI. ‍ Engineering, Operations, Sales, Legal Hiring job positions: ‍ 201-500 Company size: ‍ Hybrid, some flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program, 4% match Retirement: Unlimited PTO, 18+ company holidays PTO: ‍ $130k - $450k Salary range: ‍ No Equity available: Apply for OpenAI AI jobs → 2. Amazon is a global tech company specializing in e-commerce, cloud computing, digital streaming, and artificial intelligence (AI). A company so big, it practically needs no introduction. Founded in 1994 by Jeff Bezos as an online bookstore, Amazon is today one of the largest retailers and cloud providers (AWS) in the world, and extensively uses machine and deep learning across all its areas. Amazon As a result, both Amazon and AWS are heavily investing in AI services, especially with a new platform called Bedrock which aims to give customers access to language models to rival that of OpenAI and Google. So, Amazon is looking for software engineers, development managers, product managers, and scientists to help expand and build tools across their AI stack. ‍ Engineering, Product, Operations, Research, Administration Hiring job positions: ‍ +1,500,000 Company size: ‍ Onsite, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program, 2% match Retirement: Accrued on a per-pay-period basis PTO: ‍ $110k - $350k Salary range: ‍ Yes Equity available: Apply for Amazon AI jobs → 3. Microsoft is a technology giant that has been around since 1975. The company is best known for its Windows operating system, Office suite, and Xbox consoles. In recent years, Microsoft has invested heavily in artificial intelligence (AI) and machine learning to develop next-generation products and services. Microsoft One of Microsoft's significant partnerships in the AI space is with OpenAI, a research organization focused on developing safe AI technologies. Microsoft first invested $1 billion in OpenAI in 2019. However, this investment has since grown to $13 billion, signaling an extended partnership to build new AI models and tools. OpenAI also uses Microsoft's Azure cloud platform to develop and test its AI models. Given such a massive financial stake in developing AI, the company is actively looking to acquire the top AI talent. ‍ Engineering, Product, Design, Operations, Sales, Administration, Legal Hiring job positions: +200,000 Company size: ‍ Hybrid, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program, match up to $11,250 per year Retirement: Unlimited PTO, plus all federal holidays observed PTO: ‍ $100k - $350k Salary range: ‍ Yes Equity available: Apply for Microsoft AI jobs → 4. Grammarly is a tech startup that offers an online writing tool to help users improve their writing skills by detecting and correcting grammatical errors, spelling mistakes, and punctuation errors. Its AI-powered assistant is designed to provide reliable suggestions for enhancing the overall quality of the written content. Grammarly is a highly efficient tool that can be used across various platforms, including web browsers, desktop applications, and mobile devices. Grammarly The company has recently grown to 30 million monthly active users, and to accommodate its growing user base and expand its services, Grammarly has been actively seeking new talent in various roles. The company has been making significant efforts to acquire skilled professionals in artificial intelligence, natural language processing, and machine learning. ‍ Engineering, Design, Operations, Marketing, Sales Hiring job positions: ‍ 501-1,000 Company size: ‍ Remote-first hybrid, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program Retirement: 20 days of vacation accrual, paid sick days & holidays PTO: ‍ $160k - $300K Salary range: ‍ Yes Equity available: Apply for Grammarly AI jobs → 5. NVIDIA is a world-renowned technology company specializing in graphics processing units (GPUs) and artificial intelligence (AI). The company has been at the forefront of AI research and development for years and has made significant contributions to the field. NVIDIA's GPUs are widely used to train deep learning models, and the company has developed software and hardware solutions to optimize AI workflows. NVIDIA Just this year, NVIDIA founder and CEO Jensen Huang announced several partnerships with Google, Microsoft, and Oracle, among others, to bring new AI, simulation, and collaboration capabilities to every industry. With this massive investment in partnerships, NVIDIA is actively recruiting top AI talent to help scale its AI efforts. ‍ Engineering, Data & Analytics, Research, Sales Hiring job positions: ‍ +25,000 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program, dollar-for-dollar match every pay period (up to $6,000) Retirement: Flexible PTO, all federal holidays observed PTO: ‍ $150k - $300k+ Salary range: ‍ Yes Equity available: Apply for NVIDIA AI jobs → 6. Tessian is a tech startup that provides a cloud email security platform to intelligently protect organizations against advanced threats and data loss on email. Using machine learning and behavioral data science, Tessian aims to detect and prevent threats such as phishing, spear-phishing, and other types of social engineering attacks. They work with businesses of all sizes to protect their employees and sensitive information from cyber threats. Tessian Tessian has raised over $120 million from investors such as March Capital, Sequoia, Accel, and Balderton and is growing rapidly, hiring in dozens of engineering, design, product, marketing, sales, and operations roles. ‍ Engineering, Product, Design, Operations, Marketing, Sales Hiring job positions: ‍ 201-500 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full medical, dental, & vision Insurance: ‍ 401k program, up to 5% match Retirement: Paid holidays & sick days, 25 days annual leave PTO: ‍ $100k - $300k Salary range: ‍ Yes Equity available: Apply for Tessian AI jobs → 7. Alphabet (Google) is a global conglomerate (and parent company of Google) focusing on internet-related services, products, computer software, and hardware. Alphabet has also long been involved in developing artificial intelligence (AI) through its collaboration with DeepMind, a London-based AI research company that Google acquired in 2014. DeepMind's research focuses on developing AI systems that can learn and improve independently, without human intervention. This includes applications in machine learning, natural language processing, and robotics. Alphabet Inc Recently, in the ongoing AI race across tech, Alphabet merged DeepMind with Google Brain to accelerate progress in AI, notably prioritizing the development and release of Bard, a ChatGPT competitor. Alphabet is ramping up its acquisition of top AI talent, hiring for positions in dozens of departments. ‍ Engineering, Product, Design, Operations, Sales, Administration, Legal Hiring jobs positions: ‍ +190,000 Company size: ‍ Onsite, standard full-time hours Work policy & hours: Full health, dental, & vision benefits Insurance: ‍ 401k program, matches 50% up to 3% Retirement: Minimum 20 vacation days per year, unlimited sick days, and all federal holidays observed PTO: ‍ $100k - $300k Salary range: ‍ Yes Equity available: Apply for Alphabet AI jobs → 8. Hive is an AI startup focused on building the next generation of intelligent automation solutions. Specifically, the company offers cloud-based AI solutions for understanding content, providing developers with a portfolio of best-in-class, pre-trained AI models that serve billions of API requests per month. Hive Hive counts some of the world's largest companies as its customers, who use its AI models to help solve their challenges in content moderation, logo detection, OCR, and more. The company has raised over $120 million in funding from investors such as General Catalyst, 8VC, Tomales Bay Capital, and Glynn Capital, and is hiring in dozens of roles to support its ongoing growth. ‍ Engineering, Product, Design, Operations, Sales, Legal Hiring job positions: ‍ 51-200 Company size: ‍ Onsite, standard full-time hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ Unknown Retirement: Paid vacation PTO: ‍ $90k - $300k Salary range: ‍ Yes Equity available: Apply for Hive AI jobs → 9. IBM is a global tech company that provides hardware, software, and consulting services. In recent years, IBM has focused on artificial intelligence (AI) and has made significant strides in the field. Jeopardy! fans may already be familiar, as IBM's Watson, a natural language question-answering system, competed and beat two of the game show's greatest champions in 2011. IBM However, IBM offers many other business-grade AI products and analytics solutions designed to streamline the implementation of AI, ensure a solid data infrastructure, and prioritize positive results and ethical practices. Researchers at the company are developing the next generation of AI software and hardware advances to enterprise AI, and are actively recruiting AI talent in various roles to help them reach their goals. ‍ Engineering, Product, Design, Operations, Data & Analytics, Research, Sales Hiring job positions: ‍ +200,000 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: Full health, dental, & vision benefits Insurance: ‍ 401k program, 5% match Retirement: Accrued on a per pay-period basis PTO: ‍ $100k - $250k Salary range: ‍ Yes Equity available: Apply for IBM AI jobs → 10. Lambda is a cloud provider of GPU hardware access for deep learning that is used by dozens of Fortune 500 companies and the leading AI and machine learning researchers. Lambda's hardware and software are optimized for deep learning (one of the most computationally intensive ML techniques), such that the company bills itself as the only public cloud designed for training large language and foundation models. Lambda Lambda is still a fairly early-stage startup that has raised $44 million in a Series B round of funding. However, the company already counts engineers and researchers at the cutting edge of the AI field as customers, including companies like Intel, Microsoft, Amazon Research, Kaiser Permanente, MIT, Stanford, Harvard, Caltech, and the Department of Defense. So, Lambda is actively hiring for dozens of different roles to help support its growth. ‍ Engineering, Product, Marketing, Sales, Manufacturing Hiring job positions: ‍ 51-200 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full medical, dental, & vision Insurance: ‍ 401k program Retirement: ‍ Flexible PTO, 12 paid holidays, 5 paid sick days PTO: ‍ $100k - $250k Salary range: ‍ Yes Equity available: Apply for Lambda AI jobs → 11. Reclaim.ai is an AI-powered startup smart calendar app that optimizes and defends the schedules of busy pros and teams. The heart of the company's mission is that priorities — not random meetings — should guide your workweek, and your time should be oriented around the stuff that really matters. Reclaim automatically blocks time for your tasks, habits, meetings, and at the best time in your schedule to save up to 40% of your workweek. Reclaim.ai breaks Reclaim.ai is a rapidly growing startup backed by some of the top venture firms in the US, including Operator Partners, Index Ventures, Gradient Ventures, and Flying Fish. The AI company is looking for experienced Senior Java Engineers to help develop the smartest, fastest, conversational scheduling experience ever – along with developing backend services and APIs, evolving Reclaim's persistence layer, integrating into various collaboration platforms, and designing and deploying their cloud infrastructure. ‍ Engineers Hiring job positions: ‍ 11-50 Company size: ‍ Remote, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program Retirement: ‍ Unlimited PTO, plus all federal holidays observed PTO: ‍ $140k – $220k Salary range: ‍ Yes Equity available: Apply for Reclaim AI jobs → 12. Dataiku is a tech unicorn that provides an enterprise AI and machine learning platform for organizations to build and deploy their predictive models. Users are able to collaborate on data projects through the platform, and easily integrate their existing systems so everyone can leverage data and AI everywhere they work. Dataiku's product suite includes data preparation, data visualization, DataOps, AI/ML, and analytics tools. Dataiku Dataiku was founded in Paris in 2013 and reached the elusive unicorn status (a private company valued at more than $1 billion) in 2019. The company is backed by many heavyweight investors and partners, including CapitalG, Tiger Global, and Serena, and has raised over $840 million in funding. So, Dataiku is rapidly hiring for dozens of positions across many departments. ‍ Engineering, Product, Sales, Marketing, Operations Hiring job positions: ‍ 1,001-5,000 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full medical, dental, & vision Insurance: ‍ 401k program Retirement: 11 paid holidays, 20 vacation days, PTO: ‍ $80k - $200k Salary range: ‍ Yes Equity available: Apply for Dataiku AI jobs → 13. Accenture is a leading global professional services company that provides strategy, consulting, digital, technology, and operations services. Just this week, the company announced a $3 billion investment over the span of three years in its Data & AI practice to help its clients rapidly and responsibly advance and use AI to support their growth, efficiency, and resilience. Accenture But this investment isn’t the first time Accenture has dipped its toes in AI waters. The company holds over 1,450 AI patents and pending applications worldwide, and a portfolio of client solutions that spans a wide range of industries. And to support their new and ongoing efforts to develop and provide AI services, Accenture is currently hiring for various roles. ‍ Engineering, Design, Strategy, Operations, Marketing, Sales, Administration, Legal Hiring jobs positions: ‍ +650,000 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ 401k program, $1 for $1 match up to 6% Retirement: 17 - 27 days of PTO, 8 standard holidays, and 2 culture days PTO: ‍ $40K - $200k Salary range: ‍ Yes Equity available: Apply for Accenture AI jobs → 14. Sanctuary is a tech startup developing the world's first human-like intelligence in general-purpose robots designed to help people work more safely, efficiently, and sustainably. Their robots aim to make it possible to do a job from anywhere (without being there in person), help address labor shortages worldwide, and assist those who might be less capable of physical work. Sanctuary Being a startup focused on building learning machines, Sanctuary is actively looking for the world's best AI and ML research engineers, data scientists, software engineers, simulation experts, mechanical engineers, electronics and electrical engineers, and robotics and control engineers. ‍ Engineering, Operations, Marketing Hiring job positions: ‍ 51-200 Company size: ‍ Remote or onsite, flexible hours Work policy & hours: ‍ Full health, dental, & vision benefits Insurance: ‍ Unknown Retirement: Paid vacation, sick days, & holidays PTO: ‍ $80k - $185k Salary range: ‍ Yes Equity available: Apply for Sanctuary AI jobs → 15. Weights & Biases (W&B) is a tech startup that provides a developer-first MLOps platform that offers performance visualization tools for machine learning. As such, W&B is pretty similar to a GitHub for ML models. Their visualization tools can help companies debug, compare, and reproduce their machine learning models, whether it be architecture, hyperparameters, git commits, model weights, GPU usage, and even datasets and predictions. Weights & Biases W&B's product is widely used by AI researchers at companies like OpenAI, Lyft, Github, and MILA and has raised over $200 million in funding. So, the company is actively hiring in a number of roles to grow the product and scale. ‍ Engineering, Product, Marketing, Sales, Legal Hiring job positions: ‍ 51-200 Company size: ‍ Remote, standard full-time hours Work policy & hours: ‍ 100% Medical, Dental, & Vision Insurance: ‍ 401k program Retirement: Unlimited PTO PTO: ‍ $100k - $160k Salary range: ‍ Yes Equity available: Apply for Weights & Biases AI jobs → Shape AI's future at top companies hiring now 🤖 AI is transforming the job market much as its revolutionizing our society. From cutting-edge tech startups to established industry leaders, these companies offer exciting roles for those looking to work in the field, and there's no shortage of opportunities to learn and grow with these top AI companies. With their innovative technologies and forward-thinking cultures, they will undoubtedly shape AI's future and beyond. Did we miss any AI companies that should be on this list? What are the most innovative and exciting companies in the space? Tweet us to let us know! @reclaimai Also published . here

Top 15 AI Companies Hiring in 2023 – By Salary Range

Las 15 principales empresas de inteligencia artificial que contratan en 2023, por rango salarial

Top 15 des entreprises d'IA qui embauchent en 2023 - Par échelle de salaire

2023 में नियुक्ति करने वाली शीर्ष 15 एआई कंपनियां - वेतन सीमा के अनुसार

2023 年に雇用する AI 企業トップ 15 – 給与範囲別

As 15 principais empresas de IA contratando em 2023 – por faixa salarial

15 công ty AI hàng đầu tuyển dụng năm 2023 – Theo mức lương

2023 年招聘的 15 名人工智能公司——按薪资范围

Welcome to ’s Writing Prompts! Would you like to take a stab at answering some of these questions? The link for the template is . HackerNoon HERE Please tell us your name, what you currently do, and where you work: My name is Maggie and I am a data scientist currently working on freelance projects and writing! I previously worked as a Data Scientist for an automotive manufacturer. I was there for 3 years and started as one of the first data scientists on the team. When I started working on this team it was “back in the day”, although it was only a few years ago. So much has changed! The team I joined was a small data science team within the R&D arm of the engineering department. It was a cool opportunity to be doing data science in an R&D environment because everything was very experimental and we had a lot of freedom. We collected data from our products: big, heavy-duty, industrial trucks. These were our “end-devices” in an IoT sense. As a new, scrappy team, we operated a lot like a start-up within the large organization and this came with its own challenges. One of the big ones was that there were no data engineers. I was hired partially because I had experience as a Linux systems administrator in a data center and I had done well in my High-Performance Computing class in my master’s degree program. One of my classmates in that class was also on the team and reached out to me for my resume. I was excited to get a job where I could actually apply the big data skills that we had painfully learned in school--ie., working with Hadoop, writing stateless programs that could be distributed, writing map reduce code from scratch, and running in the cloud. I cut my teeth on building end-to-end IoT data solutions focused on predictive maintenance for fleets of vehicles. I wrote pipelines using PySpark, then later using Kafka in NiFi, picking up MQTT messages, passing them to HBase tables, then later to Kafka messaging queues. I wrote jobs to transform the raw landed data into a usable format. I designed the database structure, aggregated tables, and features to feed into my analyses, dashboards, and data products. I performed experiments with test benches and live vehicles to collect data to answer specific engineering questions about vehicle performance, part reliability, and oil life. How did you get into Data Science? I was a mathematics major during my undergraduate degree. I was required to take a semester of programming and loved it, so I took an additional semester of (C++) Object Oriented Programming. However, I was a single mom while in college. I worked full-time in addition to taking classes full-time. There were some lab-based programs that I was interested in, but labs are typically several hours a few times a week for only 1 credit. I would have needed to spend an additional year in school to get a lab-based degree and continue to work as much as I needed to support my son. So, I got my degree in mathematics as it required 0 lab work. After graduating, with a degree in mathematics and no real applied skills, I had a hard time finding a good job. I couldn’t even get an internship. The only recruiter at the job fair who called me back was the National Guard recruiter, of course. So, I joined the National Guard to help pay back my student loans and get some real job training. I became a Signal Officer in the Wisconsin National Guard and earned IT certifications that helped me get a job in IT. I worked as an IT operations analyst for an insurance company briefly before getting a dream job as a Linux system administrator in a data center for the national guard. This was in 2014-2015 and I had been hearing the word “Data Science” and getting interested. It really appealed to me because I really loved computers and math. So within a few months of starting my job in the data center, I applied for an online Master’s degree program in Data Science. I think I was among the second group of students to join that program. It was very much the early days of data science Master’s programs. It took me about 4 years to complete my Master’s degree--I was a single mom working full-time in the data center with a part-time job as an Officer in the National Guard. I took 1 class at a time and loved every minute of it. What does your day-to-day role involve? When I was working as a data scientist for a corporation, my day-to-day typically involved at least one meeting, lots of chatting online with my co-workers, some impromptu phone calls with collaborators, and lots of coding. I was always either building a pipeline, managing tables and automated jobs, or deep into Python code working on an analysis or model. I used Python primarily, but we also had licenses for JMP, which is a great tool. Eventually, we started using Dataiku and I spent most of my time in Python notebooks, Dataiku, NiFi, or Hue. What technology or programming languages do you frequently use to get your work done? Python, of course. VS Code. Pandas, Scikit-learn, and a whole toolbox of packages in Python I love Dataiku as a tool as well. What do you like the most about your job? I love the end-to-end process that includes thinking about a problem, figuring out how what kind of data you need to solve the problem, gathering the data, analyzing it, modeling it, building some kind of product or output, and then telling the story to help others see what you have done. I love the technical conversations and mathematical creativity that goes into solving the problems, but even more so, i love pulling back from the weeds of the analysis and putting together a cohesive story about the work that was done. The analysis is useless if you can’t communicate about it, and when we communicate well, the data science work becomes much more interesting and useful to everyone. I also love the people. I have never felt more at home than I have on a team of IT or data professionals. I’ve been lucky to work with people from all over the world and I think it’s a beautiful opportunity to work with people of different ages with diverse skills, cultures, etc. group of people, while at the same time geeking out over cool new topics in data science. What is something you think people do not know about data science or data scientists? I think people do not know that the field has burst open so quickly that there is no single definition or skill set of a data scientist. This can be overwhelming and lead to imposter syndrome for people who are getting started. I think it’s important to show how much variety there are in tasks, skills, applications, and points of view that goes into data science. What are some trends in data science that you’re particularly excited about? I am excited about decentralized learning. I think decentralization is very important for data ownership and privacy. It also plays a big role in the efficiency of algorithms. These are a few big challenges in the field as far as making more energy-efficient algorithms and respecting people’s data rights. I am also excited about IoT and cyber-physical systems. Through real-time data gathered from sensors, we can monitor and optimize processes to reduce waste and improve efficiency to make significant impacts on reducing carbon emissions. Additionally, IoT has applications in farming, supply chain, and weather monitoring that can potentially help us solve real human issues around the world. If you weren't a Data Scientist, what would you be doing? I would be a researcher and writer! I am currently working on building a website at www.datalabnotes.com and I am writing a book to help data scientists organize their projects. The book is equally useful for consultants, students, and seasoned professionals because it walks through the data science project lifecycle and provides a template for asking questions and taking notes along the way. It’s largely inspired by , with some added features based on recently published papers, industry best practices, and on my own experience. CRISP-DM What are 3 quick ways for someone to get into Data Science and land a job within a year? Identify the niche that you want to work in and do an analysis. Here’s how: If you are restricted by geography, look at the companies around you and companies that offer remote positions. If you are not restricted by geography, then do some soul-searching and figure out what types of problems you want to work on or what companies you want to work for. Identify what industries those companies are in, what kind of data they might have and what problems they are solving. Don’t just guess, read the job descriptions and make a spreadsheet. Start accumulating data around the types of skills and technologies listed in the job advertisements. Connect with people at these companies through LinkedIn, Kaggle, HuggingFace, and Medium blogs. Pay attention to what they are talking about and engage with them to understand what tools they are using and what kinds of problems they might be solving. Next, figure out what skills you want to focus on based on your spreadsheet and your interactions with your new network. When something new and shiny comes along, check your list of skills. If it’s not on there, don’t get distracted! Focus on building competence and skills where it matters, based on your research. Now the hard part. Get creative. Identify a project or two where you can showcase these skills. You can either build your own dataset or use open-source data. The project should be related to the projects that you observe in the target industry or at the target companies. Of course, you will need to be creative because you can’t possibly do the same work with open-source data and limited resources. You need to find smaller, related projects. You can read papers and write reviews of related work, or find a small proof of concept project. Or build a dashboard using industry data. Or build a pipeline that simulates data and a use case from the target industry. The possibilities are endless. Find a way to make a relevant project. If you are clear about the industry or problem you want to target, then a few projects like this can apply broadly to multiple companies. The third step should result in a project that you can add to your portfolio--consider using a Jekyll site on GitHub pages as a very quick and easy way to have a professional online portfolio. Even if you have Kaggle notebooks or Tableau public dashboards, create the Jekyll site as a central website to showcase all of these projects in one place. This will make it extremely easy to share and link to all of your work at once. Mine is a work in progress but here it is: . I’ve seen better, but it’s better to have a work in progress than nothing at all! https://projects.datalabnotes.com/ So now you should have 1) a clear idea of the industry/company/domain you want to work in; 2) a professional network of people related to that industry/company/domain; and 3) a portfolio relevant to the job and industry you want. Oh by the way, make sure you post and interact with your network as you are doing step 3! I think this is a solid recipe for standing out and getting some calls for interviews! Would you like to take a stab at answering some of these questions? The link for the template is just start writing! Interested in what others had to say in their answers? Click . Interested in reading the content from all of our writing prompts? Click . HERE, HERE HERE

How One Math Major's Journey Led to a Career in Data Science

Exploring the Top Data Science and Machine Learning (DSML) Platforms of 2022 

This article is a rapid introduction to Dagster using a small ML project. It is beginner-friendly but might also suit more advanced programmers if they don't know Dagster. Directed Acyclic Graph-ster Data processing systems typically span multiple runtime, storage, tooling, and organizational boundaries. But all the stages in a data processing system share a fundamental property; they are directed acyclic graphs (DAGs) of functional computations that consume and produce data assets. So, what is Dagster? Dagster is a data orchestrator for machine learning, analytics, and ETL. It lets you define in terms of the data flow between reusable, logical components, then . With a unified view of pipelines and the assets they produce, Dagster can schedule and orchestrate Pandas, Spark, SQL, or anything else that Python can invoke. It makes pipelines test locally and run anywhere testing easier and deploying faster. Why should you care? One of the core promises of Dagster is that the developer can write solids that express pure business logic, separated from concerns that are specific to the execution environment. Code that's structured in this way is easier to develop and test locally, swapping heavy production dependencies out for lighter test systems or even fixtures and mocks. This leads to dramatically shorter feedback loops for developers, which in turn leads to order-of-magnitude improvements in productivity. Code written in this way is also easier to reuse in different contexts and more portable when it becomes necessary to change out a third-party system. The fundamentals To keep the focus of this article on jump-starting a project, you can read a very brief overview of 4 fundamental elements of Dagster here: https://docs.dagster.io/tutorial/overview Then, let’s have a look at a  simple ML program. The application To keep it simple, let’s create a with our old pal sci-kit. text classifier The code as well as further explanation to execute the code can be found at https://github.com/stephanBV/ML_with_DAGs A solid is a unit of computation that yields a stream of events (similar to a function), while a composite is a collection of solids. Our script creates a single pipeline which: processes the data searches for optimal parameters between a logistic regression and a random forest trains and tests the models The raw data is a collection comments from a social media platform, each comment has been labeled ‘positive’ or ‘negative’. The data Once we launch the Dagster’s UI, we get the following DAG: Dagster in action We can see each solid in light blue and how they are connected to each other, as well as the composites in light purple. NB: Each solid also has an ‘INFO” section which describes its inputs and outputs. Furthermore, each composite has an “ ” button, which let you see the set of solids it contains: Expand Here, you can see that the composite “ ” contains the grid search, training, and test solids. process_model At the top, you can also see three inputs: result, train, and test, which are the outputs of the solids “prepare_grid_random_forest”, “split_data” with the train data, and “split_data” with the test data. Parameters can be in a YAML file in your repository or can be added to the , as follow: Playground section Playground also checks your parameters against your code in and will let you know instantly of any errors found. real time If everything is okay, you will see at the bottom right of the window that the name of the solids with the parameters above are in a light-blue colour and the other solids are in grey colour. As long as nothing is red, and you can launch an execution. you are good to go Once the pipeline has finished executing, the Runs section provides the list of all executions, as follow. You can access the detail of all execution by clicking on its id (e.g. b29a71b4). It provides a graphical representation of the execution duration - -as well as a table which details all the steps that happened during the process. An important part on that table is the which is particularly useful, especially if you are running on DEBUG mode. step_logs For example, the following is the output of the step_logs from the solid that tests the logistic regression. Conclusion To keep this article simple, only one pipeline was represented, although you can have as many pipelines as necessary. One could be specific to mine the data, another to process the data, another to test the code (with Pytest for e.g.), another could train, test, etc.. Dagster is an amazing tool to experiment, monitor, and maintain your ML pipeline, It kind of makes me think of an Open Source version of Azure ML Studio and Dataiku, but you get much more control over your project. We only scratched the surface of Dagster. For more information, visit / https://dagster.io

A Quick Introduction to Machine Learning with Dagster

Some systems are specifically meant to be hacked. The little experiment I describe in this article shows that a system is found quite soon after being connected to the Internet. Security experts tell us that external access to servers (directly connecting them to the Internet) . However, security departments sometimes deliberately set up systems to be targeted by attackers. These so-called are used to uncover possible hacker activity going on in the companies network. Besides this business use, a similar setup can be done out of interest or for fun as well. is insecure honeypots A commonly used protocol used to connect to computer systems is . Whether it is safe to connect it to the open internet, even when taking specific precautions, . But how long will it take before such a system will be compromised? To find out, I deployed the honeypot on an Amazon EC2 instance, with the mocked ssh service on port 2222 directly reachable from the Internet. ssh is disputed Cowrie SSH As nothing on the Internet is referring to this server, it can only be found by scanning IP and port ranges. Using a signed SSL certificate with a domain name would probably make it easier to . However, as the image below shows, it took only a few hours before the first connection attempt arrived at our exposed ssh port. find specific targeted attacks The chart shows the source IP address (unreadable) on the vertical axis and the moment of first connection on the horizontal axis. The size of the dot is relative to the number of connection attempts from that specific source address. The total amount of connection attempts was over 3,600 during these five days, originating from 37 different source addresses. The map below shows the location of the source addresses, according to . Again, the size of the dot corresponds to the number of connection attempts from that location. We see connections are coming in from all over the world. However, chances to find our hacker at that location are small, as VPN and other obfuscation techniques are widely used when misbehaving on the Internet. ipstack The honeypot SSH server is protected by username/password credentials. Several commonly used username/password combinations are granting access. Once logged in a shell is emulated, allowing commands to be entered, imitating the command result. During the testing period, three attackers managed to pass the login prompt by guessing the right password. Once connected the intruders were using the command to check the type of operating system and version. None of the intruders came back for more. uname Considering this, I would think twice before connecting any server directly to the Internet. It doesn't take long before the first automated hacker-operated scanner comes by to knock on your ports. Then you'd better make sure to keep them closed! Lead image by Magda Ehlers@Pexels, other images were created from log analysis using Dataiku.

A Honeypot Experiment Shows How Long It Takes To Get Hacked

One of the known truths of the Machine Learning(ML) world is that it takes a lot longer to deploy ML models to production than to develop it.¹ The problem of deploying ML models to production is well known. Modern software requires a variety of crucial properties such as on-demand scaling and high availability. As a result, it might take a lot of effort and time to correctly deploy models into productions. Let’s discuss some different options you have when it comes to deploying ML models. Variants are provided in order from the most general to ML-specific. 1. Hardwave / VM The most way to deploy anything is to rent a VM, wrap a model into some kind of a server and leave it running. While being extremely straightforward and customizable this method has numerous drawbacks such as hard integration into CI/CD pipelines and isolation problems. direct 2. Containers It is possible to deploy ML models in Docker containers using Kubernetes or similar orchestration tools. This option provides way more quality of life improvements. Models can be easily wrapped into specially designed servers such as or (works for VM option as well). Now it is even easier to chain models together using highly sophisticated frameworks such as . NVIDIA Triton Tensorflow Runtime Kubeflow However, customizability comes at a cost of DevOps complexity and a requirement to maintain technologies that make your model run. 3. General purpose serverless platforms An easy way to just drop your model on the cloud would be using serverless PaaS platforms. Here you have to wrap your model into some preprocessing and postprocessing code. Platforms like or provide more flexibility since you can even wrap your code into a container while functions, or make it much easier to deploy, even providing great integrations into respective cloud services. Heroku Google App Engine AWS Lambda Google Cloud Functions Azure Function This approach is great for background task processing since inference time is relatively high because you are and models themself are commonly stored from processing nodes and may require time to load. limited to processing models on CPU far away 4. ML-focused serverless providers Now we are seeing a rise of ML-focused serverless providers that host your model providing an API or a set of frameworks. One set of providers would be or . These services still require you to rent underlying compute instances on which your models will be running. Amazon SageMaker Google Cloud AI Platform Another option would be to use , , . Here you generally get serverless experience since you pay only for time your models are running. DeepMux Algorithmia Dataiku true In general, using ML-focused serverless providers allows you to separate GPU-intensive computations from CPU-intensive while providing on-demand scalability for the former. However, you still have to perform pre and post-processing on the client application or using cloud functions. That's it We successfully reviewed common options to deploy ML models. Thank you for reading! Stay tuned for more articles and feel free to write in the comment section or ask questions on msitnikov@deepmux.com One of the biggest underrated challenges in machine learning development is the deployment of the trained models in production that too in a scalable way. One joke on it I have read is “Most common way, Machine Learning gets deployed today is powerpoint slides :)”. References Adarsh Shah. (June 21, 2020). Challenges Deploying Machine Learning Models to Production https://towardsdatascience.com/challenges-deploying-machine-learning-models-to-production-ded3f9009cb3 Sambit Mahapatra. (March 17, 2019). Machine Learning Models as Micro Services in Docker https://towardsdatascience.com/machine-learning-models-as-micro-services-in-docker-a798e1f068a5 James Le.(March 6, 2020). The 5 Components Towards Building Production-Ready Machine Learning Systems https://medium.com/cracking-the-data-science-interview/the-5-components-towards-building-production-ready-machine-learning-system-a4d5237ec04e Also published at https://medium.com/@deepmux/deploying-ml-models-into-production-with-ease-8fbe48bc10e8

Stories, News & Mentions about DATAIKU

Dataiku’s Stories on HackerNoon

Dataiku's News & Mentions