Hundreds of thousands of companies today use webinars and physical events for thought leadership, promoting products, and strengthening customer relationships

Contentware

TL;DR 💨 Contentlayer reduces a lot of friction in publishing the content on my static website. 😍 MDX with remark and rehype plugins is awesomely convenient and powerful. 🍵 Next.js jells very well with Contentlayer and MDX I fully embraced (SSG) for my website to optimize the site speed and scaling in the future. Static Site Generation Recently I’ve been researching on reducing the effort to create a new article on my website. There were many touch points in my project setup in order to: Next.js Publish a new article Estimate reading time Update the list of articles Create a new RSS feed Update the sitemap What Was The Problem? I analyzed a little more and found out the friction was in my MDX usage. My file structure looked like this: my-blog
├── public
├── data
│   └── blogs.json
├── components
│   └── Blog.tsx
└── pages
    ├── blogs
    │   ├── blog-one.mdx
    │   ├── blog-two.mdx
    │   └── blog-three.mdx
    └── index.tsx It is the recommended by Next.js. I was using and to transform MDX into pages. standard setup @mdx-js/loader @next/mdx Take for example, the content looked like this: pages/blogs/blog-one.mdx pages/blogs/blog-one.mdx import Blog from '../../components/Blog'

export const meta = {
  title: 'Blog One🚀',
  publishedAt: 'February 4, 2022'
  description: "Learn how to build a Next.js blog with MDX and Contentlayer!",
  cover: '/optimized/articles/blog-one/hero.webp',
}

export default ({ children }) => (
  <Blog
    title={meta.title}
    description={meta.description}
    cover={meta.cover}
    publishedAt={meta.publishedAt}
  >
    {children}
  </Blog>
)

Hey There👋

Welcome to Blog one✨ Let's learn together! named-exported a meta data. It was picked up by the the default component that took care of the layout and rendered the meta data. blog-one.mdx The component looked like this: <Blog /> components/Blog.tsx import { BlogProps } from './types'

export default function Blog(props: BlogProps) {
  return (
    <article>
      <h1>{props.title}</h1>
      <h2>{props.description}</h2>
      <p>
        {props.publishedAt}
      </p>
      <img alt={props.title} src={props.cover} width="100%" loading="lazy" />
      {props.children}
    </article>
  )
} I was treating MDX files as pages. Because the data in each MDX file was trapped in the page, I duplicated all the meta data and aggregated them in . I used it to maintain the list of articles on my website, the RSS feed, and the for SEO. meta data/blogs.json sitemap It would be much better if I could treat the MDX files as data, and generate pages based on the data. This way, . Publishing a new article ideally could be much more frictionless. I could use the MDX files as data points and page content at the same time I came across and found out he was using a alpha library called to solve the problem. Lee Robinson’s website Contentlayer What is Contentlayer is an library in its early stage that turns content into data. It works roughly like this: Contentlayer It takes in headless CMS or local content in YAML, JSON, MDX, or Markdown as source. It transforms the content into TypeScript types and data files in JSON that includes the original content, meta data, and any derived data we specified. It aggregates all the data in JSON and exports them as . ESM For my use case, I can use the aggregated data generated by Contentlayer to replace my previous manual process: I use the generated data to build the new page for an article. I use the generated data to render . the list of articles I use the generated data to create a new RSS feed. I use the new file structure to generate a new sitemap. All automatic! Contentlayer offers easy integration with Next.js. I’ll show you how in the next sections. Using MDX as Data Let’s first explore how to use MDX as a data point. MDX offers support with custom parsers. You can express the meta data like this: YAML frontmatter ---
title: 'Blog One🚀'
publishedAt: 'February 4, 2022'
description: 'Learn how to build a Next.js blog with MDX and Contentlayer!'
cover: '/optimized/articles/blog-one/hero.webp'
---

Hey There👋

Welcome to Blog One✨ Let's learn together! You can see the meta data in YAML syntax is inside the block, and the body of the content follows in MDX syntax. Compared to the where MDX files were treated as pages, the new MDX file contains only meta data and content. --- old setup The next thing we need to do is to generate the blog page that renders the meta data and the content with the layout from component. <Blog /> Integrating Contentlayer in Next.js Now that we updated the MDX files to contain only data and content, Let’s move them into the directory. data The new file structure looks like this: my-blog
├── public
├── components
│   └── Blog.tsx
├── pages
│   ├── blogs
│   │   └── [slug].tsx
│   └── index.tsx
└── data
    └──blogs
       ├── blog-one.mdx
       ├── blog-two.mdx
       └── blog-three.mdx Notice that we replaced the MDX files in directory with a . We'll use this page to statically generate the blog pages . pages/blogs dynamic route [slug].tsx later Configuring Contentlayer Contentlayer offers seamless integration with Next.js. To install the dependencies: yarn add contentlayer next-contentlayer Contentlayer reads the configuration from . Let's create one. contentlayer.config.ts touch contentlayer.config.ts Inside the , we need to add instructions to tell Contentlayer how to parse: contentlayer.config.ts : namespace name : input files filePathPattern : content body type for parsing bodyType : meta data fields fields : derived meta data fields computedFields contentlayer.config.ts import { defineDocumentType, makeSource } from 'contentlayer/source-files'
import readingTime from 'reading-time'

export const Blog = defineDocumentType(() => ({
  name: 'Blog',
  filePathPattern: 'blogs/*.mdx',
  bodyType: 'mdx',
  fields: {
    title: { type: 'string', required: true },
    publishedAt: { type: 'string', required: true },
    description: { type: 'string', required: true },
    cover: { type: 'string', required: true },
  },
  computedFields: {
    readingTime: { type: 'json', resolve: (doc) => readingTime(doc.body.raw) },
    slug: {
      type: 'string',
      resolve: (doc) => doc._raw.sourceFileName.replace(/\.mdx/, ''),
    },
  },
}))

export default makeSource({
  contentDirPath: 'data',
  documentTypes: [Blog],
  mdx: {
    remarkPlugins: [],
    rehypePlugins: [],
  },
}) In the , we can compute data like from the content body🤩. I'm using for calculating the reading time based on word count. The field is for generating the dynamic route later in the page. computedFields readingTime reading-time slug [slug].tsx Under the hood, Contentlayer uses to parse MDX and YAML frontmatter and extract the content and data. If you're interested in the magic behind it, you can read more about and . These are the libraries uses internally. mdx-bundler gray-matter remark-mdx-frontmatter mdx-bundler At the end of the configuration, will then look for files that match pattern under directory and generate the blog data in directory at your project root. makeSource blogs/*.mdx data .contentlayer Lastly, wrap your Next.js configuration with to integrate with Next.js's live-reload and build process. next-contentlayer next.config.js const { withContentlayer } = require('next-contentlayer')

module.exports = withContentlayer()({
  // ... your Next.js config
}) Using Contentlayer data for Static Site Generation We are ready to use the generated data from Contentlayer and build the static pages🤩 All we need to do is to use from to build the dynamic routes with and use to pass the blog data to the page. allBlogs .contentlayer/data getStaticPaths getStaticProps [slug].tsx pages/blogs/[slug].tsx import { useMDXComponent } from 'next-contentlayer/hooks'
import { allBlogs } from '.contentlayer/data'
import type { Blog } from '.contentlayer/types'
import BlogLayout from '../../../components/Blog'

type BlogProps = {
  blog: Blog
}

export default function Blog({ blog }: BlogProps) {
  const Component = useMDXComponent(post.body.code)

  return (
    <BlogLayout {...blog}>
      <Component />
    </BlogLayout>
  )
}

export async function getStaticPaths() {
  return {
    paths: allBlogs.map((blog) => ({ params: { slug: blog.slug } })),
    fallback: false,
  }
}

export async function getStaticProps({ params }) {
  const blog = allBlogs.find((blog) => blog.slug === params.slug)
  return { props: { blog } }
} After the project is built, you’ll see the blogs available at , , and ✨ /blogs/blog-one /blogs/blog-two /blogs/blog-three Bonus: remark & rehype Plugins There are a lot more we can do with MDX by leveraging remark and rehype plugins in the . contentlayer.config.ts is an awesome plugin ecosystem that transforms markdown. remark is another powerful plugin ecosystem that transforms HTML. rehype They are two separate ecosystems but we can convert remark to rehype and generate HTML markup. The transformation looks like this: MDX ----> remark AST ------> rehype AST --------> HTML
    parse            convert            stringify Contentlayer takes care of the flow. All we need to do is add the plugins to provide instruction for the transformations. I’m using the following plugins: to support . remark-gfm GitHub Flavored Markdown and to render heading links. rehype-slug rehype-autolink-headings to render syntax highlighting in code blocks. rehype-prism-plus to render code block titles. rehype-code-titles to provide accessibility to emojis. rehype-accessible-emojis contentlayer.config.ts + import remarkGfm from 'remark-gfm'
+ import rehypeSlug from 'rehype-slug'
+ import rehypeAutolinkHeadings from 'rehype-autolink-headings'
+ import rehypeCodeTitles from 'rehype-code-titles'
+ import rehypePrism from 'rehype-prism-plus'
+ import { rehypeAccessibleEmojis } from 'rehype-accessible-emojis'

// ...

export default makeSource({
  mdx: {
-    remarkPlugins: [],
+    remarkPlugins: [remarkGfm],
-    rehypePlugins: [],
+    rehypePlugins: [
+      rehypeSlug,
+      rehypeCodeTitles,
+      rehypePrism,
+      rehypeAutolinkHeadings,
+      rehypeAccessibleEmojis,
    ],
  },
}) Contentlayer Applications There are more things we can do with the data. Application #1: RSS Feed I can now write a script to generate an RSS feed base on the data! allBlogs scripts/rss.mjs import { writeFileSync } from 'fs'
import RSS from 'rss'
import { allBlogs } from '.contentlayer/data'

const feed = new RSS({
  title: "My Blogs",
  feed_url: 'localhost:3000/rss.xml',
  site_url: 'localhost:3000',
})

allBlogs.map((blog) => ({
  title: blog.title,
  description: blog.description,
  url: `localhost:3000/blogs/${blog.slug}`
  date: blog.publishedAt,
})).forEach((item) => {
  feed.item(item)
})

writeFileSync('./public/rss.xml', feed.xml({ indent: true })) Application #2: XML Sitemap It’s easier to write a script for sitemap generation. All we need is the file structure in the and directories. data page scripts/sitemap.mjs import { writeFileSync } from 'fs'
import { globby } from 'globby'
import prettier from 'prettier'

const pages = await globby([
  'pages/*.tsx',
  'data/**/*.mdx',
  '!pages/_*.tsx',
])

const urlTags = pages
  .map((file) =>
    file
      .replace('pages', '')
      .replace('data/content', '')
      .replace('.tsx', '')
      .replace('.mdx', '')
  )
  .map((path) => (path === '/index' ? '/' : path))
  .map(
    (path) => `
      <url>
          <loc>localhost:3000${path}</loc>
      </url>
    `
  )
  .join('')

const sitemap = `
  <?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
      ${urlTags}
  </urlset>
`

const prettierConfig = await prettier.resolveConfig('./prettierrc')
const formatted = prettier.format(sitemap, {
  ...prettierConfig,
  parser: 'html',
})

writeFileSync('public/sitemap.xml', formatted) Run both scripts after building the project and automatically generate a new RSS feed and sitemap. In , add: package.json "scripts": {
+    "sitemap": "node scripts/sitemap.mjs",
+    "rss": "node scripts/rss.mjs",
+    "postbuild": "yarn sitemap && yarn rss",
  }, Final Thoughts Building Static sites with Next.js becomes so effortless with MDX and Contentlayer. MDX combines with remark and rehype ecosystem enriches the possibility of building consistent pages with Markdown. Contentlayer makes data and content in MDX files available to consume in Next.js projects. If you’re exploring ways to build your own static sites, check out the libraries mentioned earlier. It not only reduces the time to market but also is a lot of fun to build!🦄 This article is originally posted on . Daw-Chih’s website

Build Faster Next.js Static Sites Using MDX And Contentlayer 

Build a Dynamic Photo Collection App With Django

Authors:
(1) Rui Cao, Singapore Management University;
(2) Ming Shan Hee, Singapore University of Design and Technology;
(3) Adriel Kuek, DSO National Laboratories;
(4) Wen-Haw Chong, Singapore Management University;
(5) Roy Ka-Wei Lee, Singapore University of Design and Technology
(6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Related Work Preliminary Proposed Method Experiment Conclusion and References Appendix ABSTRACT Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from noninformative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot visual question answering (VQA) manner. Specifically, we prompt a frozen PVLM by asking hateful contentrelated questions and use the answers as image captions (which we call Pro-Cap), so that the captions contain information critical for hateful content detection. The good performance of models with Pro-Cap on three benchmarks validates the effectiveness and generalization of the proposed method.[1] CCS CONCEPTS • Computing methodologies → Natural language processing; Computer vision representations. KEYWORDS memes, multimodal, semantic extraction ACM Reference Format: Rui Cao, Ming Shan Hee, Adriel Kuek, Wen-Haw Chong, Roy Ka-Wei Lee, and Jing Jiang. 2023. Pro Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29-November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3581783.3612498 Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. 1 INTRODUCTION Memes, which combine images with short texts, are a popular form of communication in online social media. Internet memes are often intended to express humor or satire. However, they are increasingly being exploited to spread hateful content across online platforms. Hateful memes attack individuals or communities based on their identities such as race, gender, or religion [5, 8, 12, 27]. The propagation of hateful memes can lead to discord online and may potentially result in hate crimes. Therefore, it is urgent to develop accurate hateful meme detection methods. The task of hateful meme detection is challenging due to the multimodal nature of memes. Detection involves not only comprehending both the images and the texts but also understanding how these two modalities interact. Previous work [14, 28, 35, 36] learns cross-modal interactions from scratch using hateful meme detection datasets. However, it may be difficult for models to learn complicated multimodal interactions with the limited amount of data available from these datasets. With the development of Pretrained Vision-Language Models (PVLMs) such as VisualBERT [18] and ViLBERT [23], recent work leverage these powerful PVLMs to facilitate the hateful meme detection task. A common approach is to fine-tune PVLMs with task-specific data [9, 20, 26, 34, 37]. However, it is less feasible to fine-tune the larger models such as BLIP-2 [15] and Flamingo [1] on meme detection because there are billions of trainable parameters. Therefore, computationally feasible solutions other than direct fine-tuning are needed to leverage large PVLMs in facilitating hateful meme detection. Different from the approach above using PVLMs, PromptHate[2] is a recently proposed model that converts the multimodal meme detection task into a unimodal masked language modeling task. It first generates meme image captions with an off-the-shelf image caption generator, ClipCap [25]. By converting all input information into text, it can prompt a pre-trained language model along with two demonstrative examples to predict whether or not the input is hateful by leveraging the rich background knowledge in the language model. Although PromptHate achieves state-of-the-art performance, it is significantly affected by the quality of image captions, as shown in Table 1. Image captions that are merely generic descriptions of images may omit crucial details [14, 37], such as the race and gender of people, which are essential for hateful content detection. But with additional image tags, such as entities found in the images and demographic information about the people in the images, the same model can be significantly improved, as shown in Table 1. However, generating these additional image tags is laborious and costly. For instance, entity extraction is usually conducted with the Google Vision Web Entity Detection API [2], which is a paid service. Ideally, we would like to find a more affordable way to obtain entity and demographic information from the images that is critical for hateful content detection. Both above-mentioned approaches (i.e., one using PVLMs and the other converting the task to a unimodal task) have their pros and cons. In this paper, we combine the ideas from these two approaches and design a hateful meme detection method that leverages the power of a frozen PVLM to complement the unimodal approach of PromptHate. Specifically, we use a set of “probing” questions to query a PVLM (BLIP-2 [15] in our experiments) for information related to common vulnerable targets in hateful content. The answers obtained from the probing questions will be treated as image captions (denoted as Pro-Cap) and used as input to a trainable hateful meme detection model. Figure 1 illustrates the overall workflow of the method. We refer to the step of using probing questions to generate the captions as probing-based captioning. Our proposed method fills existing research gaps by: 1) Leverage a PVLM without any adaptation or fine-tuning, thereby reducing computational cost; 2) Instead of explicitly obtaining additional image tags with costly APIs, we utilize the frozen PVLM to generate captions that contain information useful for hateful meme detection. To the best of our knowledge, this is the first work that to leverage PVLMs in a zero-shot manner through question answering to assist in the hateful meme detection task. To further validate our method, we test the effect of the generated Pro-Cap on both PromptHate[2] and a BERT-based[4] hateful meme detection model. Based on the experimental results, we observe that PromptHate with Pro-Cap (denoted as Pro-CapPromptHate) significantly surpasses the original PromptHate without additional image tags (i.e., about 4, 6, and 3 percentage points of absolute performance improvement on FHM [12], MAMI [5], and HarM [28] respectively). ProCapPromptHate also achieves comparable results with PromptHate with additional image tags, indicating that probing-based captioning can be a more affordable way of obtaining image entities or demographic information. Case studies further show that Pro-Cap offers essential image details for hateful content detection, enhancing the explainability of models to some extent. Meanwhile, ProCapBERT clearly surpasses multimodal BERT-based models of similar sizes (i.e., about 7 percentage points of absolute improvement with VisualBERT on FHM [12]), proving the generalization of the probing-based captioning method. [1] Code is available at: https://github.com/Social-AI-Studio/Pro-Cap [2] https://cloud.google.com/vision/docs/detecting-web This paper is available on arxiv under CC 4.0 license. Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Authors: Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Preliminary Preliminary Proposed Method Proposed Method Experiment Experiment Conclusion and References Conclusion and References Appendix Appendix ABSTRACT Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from noninformative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot visual question answering (VQA) manner. Specifically, we prompt a frozen PVLM by asking hateful contentrelated questions and use the answers as image captions (which we call Pro-Cap), so that the captions contain information critical for hateful content detection. The good performance of models with Pro-Cap on three benchmarks validates the effectiveness and generalization of the proposed method.[1] CCS CONCEPTS • Computing methodologies → Natural language processing; Computer vision representations. • Computing methodologies → Natural language processing; Computer vision representations. KEYWORDS memes, multimodal, semantic extraction ACM Reference Format: Rui Cao, Ming Shan Hee, Adriel Kuek, Wen-Haw Chong, Roy Ka-Wei Lee, and Jing Jiang. 2023. Pro Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29-November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3581783.3612498 Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. 1 INTRODUCTION Memes, which combine images with short texts, are a popular form of communication in online social media. Internet memes are often intended to express humor or satire. However, they are increasingly being exploited to spread hateful content across online platforms. Hateful memes attack individuals or communities based on their identities such as race, gender, or religion [5, 8, 12, 27]. The propagation of hateful memes can lead to discord online and may potentially result in hate crimes. Therefore, it is urgent to develop accurate hateful meme detection methods. The task of hateful meme detection is challenging due to the multimodal nature of memes. Detection involves not only comprehending both the images and the texts but also understanding how these two modalities interact. Previous work [14, 28, 35, 36] learns cross-modal interactions from scratch using hateful meme detection datasets. However, it may be difficult for models to learn complicated multimodal interactions with the limited amount of data available from these datasets. With the development of Pretrained Vision-Language Models (PVLMs) such as VisualBERT [18] and ViLBERT [23], recent work leverage these powerful PVLMs to facilitate the hateful meme detection task. A common approach is to fine-tune PVLMs with task-specific data [9, 20, 26, 34, 37]. However, it is less feasible to fine-tune the larger models such as BLIP-2 [15] and Flamingo [1] on meme detection because there are billions of trainable parameters. Therefore, computationally feasible solutions other than direct fine-tuning are needed to leverage large PVLMs in facilitating hateful meme detection. Different from the approach above using PVLMs, PromptHate[2] is a recently proposed model that converts the multimodal meme detection task into a unimodal masked language modeling task. It first generates meme image captions with an off-the-shelf image caption generator, ClipCap [25]. By converting all input information into text, it can prompt a pre-trained language model along with two demonstrative examples to predict whether or not the input is hateful by leveraging the rich background knowledge in the language model. Although PromptHate achieves state-of-the-art performance, it is significantly affected by the quality of image captions, as shown in Table 1. Image captions that are merely generic descriptions of images may omit crucial details [14, 37], such as the race and gender of people, which are essential for hateful content detection. But with additional image tags, such as entities found in the images and demographic information about the people in the images, the same model can be significantly improved, as shown in Table 1. However, generating these additional image tags is laborious and costly. For instance, entity extraction is usually conducted with the Google Vision Web Entity Detection API [2], which is a paid service. Ideally, we would like to find a more affordable way to obtain entity and demographic information from the images that is critical for hateful content detection. Both above-mentioned approaches (i.e., one using PVLMs and the other converting the task to a unimodal task) have their pros and cons. In this paper, we combine the ideas from these two approaches and design a hateful meme detection method that leverages the power of a frozen PVLM to complement the unimodal approach of PromptHate. Specifically, we use a set of “probing” questions to query a PVLM (BLIP-2 [15] in our experiments) for information related to common vulnerable targets in hateful content. The answers obtained from the probing questions will be treated as image captions (denoted as Pro-Cap) and used as input to a trainable hateful meme detection model. Figure 1 illustrates the overall workflow of the method. We refer to the step of using probing questions to generate the captions as probing-based captioning. Our proposed method fills existing research gaps by: 1) Leverage a PVLM without any adaptation or fine-tuning, thereby reducing computational cost; 2) Instead of explicitly obtaining additional image tags with costly APIs, we utilize the frozen PVLM to generate captions that contain information useful for hateful meme detection. To the best of our knowledge, this is the first work that to leverage PVLMs in a zero-shot manner through question answering to assist in the hateful meme detection task. To further validate our method, we test the effect of the generated Pro-Cap on both PromptHate[2] and a BERT-based[4] hateful meme detection model. Based on the experimental results, we observe that PromptHate with Pro-Cap (denoted as Pro-CapPromptHate) significantly surpasses the original PromptHate without additional image tags (i.e., about 4, 6, and 3 percentage points of absolute performance improvement on FHM [12], MAMI [5], and HarM [28] respectively). ProCapPromptHate also achieves comparable results with PromptHate with additional image tags, indicating that probing-based captioning can be a more affordable way of obtaining image entities or demographic information. Case studies further show that Pro-Cap offers essential image details for hateful content detection, enhancing the explainability of models to some extent. Meanwhile, ProCapBERT clearly surpasses multimodal BERT-based models of similar sizes (i.e., about 7 percentage points of absolute improvement with VisualBERT on FHM [12]), proving the generalization of the probing-based captioning method. [1] Code is available at: https://github.com/Social-AI-Studio/Pro-Cap [2] https://cloud.google.com/vision/docs/detecting-web This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection

প্রো-ক্যাপ: বিদ্বেষপূর্ণ মেম সনাক্তকরণের জন্য একটি হিমায়িত দৃষ্টি-ভাষা মডেল ব্যবহার করা

Pro-Cap: Nutzung eines Frozen Vision-Language-Modells zur Erkennung hasserfüllter Memes

Pro-Cap: Aprovechamiento de un modelo de visión-lenguaje congelado para la detección de memes de odio

Pro-Cap : exploiter un modèle de langage de vision figé pour la détection des mèmes haineux

प्रो-कैप: घृणास्पद मीम का पता लगाने के लिए फ्रोज़न विज़न-लैंग्वेज मॉडल का लाभ उठाना

Pro-Cap: 凍結された視覚言語モデルを活用して憎悪的なミームを検出する

Pro-Cap: 혐오스러운 밈 탐지를 위해 고정된 비전-언어 모델 활용

Pro-Cap: aproveitando um modelo de linguagem de visão congelada para detecção de memes odiosos

Pro-Cap: использование модели Frozen Vision-Language для обнаружения ненавистных мемов

Pro-Cap: Nefret Dolu Memlerin Tespiti için Donmuş Görüş-Dil Modelinden Yararlanmak

Pro-Cap: Tận dụng mô hình ngôn ngữ tầm nhìn đông lạnh để phát hiện meme đáng ghét

Pro-Cap：利用冻结视觉语言模型进行仇恨模因检测

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the DALL-E model for image generation. Introduction The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions and apps. Prerequisite Experience working with Azure and Azure portals. An understanding of generative AI. Experience in one high-level programming language like C# or Python Steps to Create Open AI Service on Azure with “Dall-E” model deployed. Day 1 — Azure Open AI Challenge Getting Started Considering Azure Open AI Service is running on the Azure portal and the DALL-E model is deployed successfully. Step 1: Create a Console Application To test image generation, create a console application in Visual Studio or Visual Studio Code. dotnet new console Step 2: Read the Configuration Read the configuration from the appsettings.json file IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
    IConfigurationRoot configuration = builder.Build();
    string? aoaiEndpoint = configuration["AzureOAIEndpoint"] ?? "";
    string? aoaiKey = configuration["AzureOAIKey"] ?? ""; Step 3: Generate Console Prompt Get the user prompt on the console ready for image generation Console.Clear();
    Console.WriteLine("Enter a prompt to request an image:");
    string prompt = Console.ReadLine() ?? ""; Step 4: Call the DALL-E model using HttpClient Finally, Call the model using the HTTP client, and retrieve the blob URL from the API response as shown below. using (var client = new HttpClient())
    {
        var contentType = new MediaTypeWithQualityHeaderValue("application/json");
        var api = "openai/deployments/test-dall-e-3/images/generations?api-version=2024-02-15-preview";
        client.BaseAddress = new Uri(aoaiEndpoint);
        client.DefaultRequestHeaders.Accept.Add(contentType);
        client.DefaultRequestHeaders.Add("api-key", aoaiKey);
        var data = new
        {
            prompt = prompt,
            n = 1,
            size = "1024x1024"
        };
    
        var jsonData = JsonSerializer.Serialize(data);
        var contentData = new StringContent(jsonData, Encoding.UTF8, "application/json");
        var response = await client.PostAsync(api, contentData);
    
        // Get the revised prompt and image URL from the response
        var stringResponse = await response.Content.ReadAsStringAsync();
        JsonNode contentNode = JsonNode.Parse(stringResponse)!;
        JsonNode dataCollectionNode = contentNode!["data"];
        JsonNode dataNode = dataCollectionNode[0]!;
        JsonNode revisedPrompt = dataNode!["revised_prompt"];
        JsonNode url = dataNode!["url"];
        Console.WriteLine(revisedPrompt.ToJsonString());
        Console.WriteLine(url.ToJsonString().Replace(@"\u0026", "&"));
    
    } Test Run 1 Now, add the following prompt to the console window. aliens in Punjabi attire The console application will generate the following response and the user has to open the Blob URL in any browser. "A fascinating scene showing a group of extraterrestrial creatures donning traditional Punjabi attire. The aliens each have their own unique physical characteristics, yet their outfits are distinctively Punjabi. Imagine elaborate turbans, vibrant salwar kameez, and sparkling bangles. To put an interesting spin on the scene, some of the aliens are participating in traditional Punjabi activities like performing bhangra and carrying a jug of lassi. The background is a typical Punjabi village with interesting architectural features that seem to blend into an alien landscape."
    "<URL will be here>" Output Test Run 2 Now, add the following prompt to the console window. a cartoon character playing circket in the football field The console application will generate the following response and the user has to open the Blob URL in any browser. "A cartoon character with spiky blue hair and emerald green eyes, wearing a traditional cricket uniform consisting of a white shirt, trousers, and a red cricket cap. This character is striking a cricket ball with a willow bat in the middle of a vast, well-maintained football field. The field is vibrant green and has white line markings, with two goal posts at each end. The sky above is a cheerful cerulean with a few fluffy cumulus clouds scattered about."
    "<URL will be here>" Output Complete Code Please find below the complete code in the Program.cs file. using System.Net.Http.Headers;
    using System.Text.Json.Nodes;
    using System.Text.Json;
    using System.Text;
    using Microsoft.Extensions.Configuration;
    
    try
    {
        // Get config settings from AppSettings
        IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
        IConfigurationRoot configuration = builder.Build();
        string? aoaiEndpoint = configuration["AzureOAIEndpoint"] ?? "";
        string? aoaiKey = configuration["AzureOAIKey"] ?? "";
    
        // Get prompt for image to be generated
        Console.Clear();
        Console.WriteLine("Enter a prompt to request an image:");
        string prompt = Console.ReadLine() ?? "";
    
        // Call the DALL-E model
        using (var client = new HttpClient())
        {
            var contentType = new MediaTypeWithQualityHeaderValue("application/json");
            var api = "openai/deployments/test-dall-e-3/images/generations?api-version=2024-02-15-preview";
            client.BaseAddress = new Uri(aoaiEndpoint);
            client.DefaultRequestHeaders.Accept.Add(contentType);
            client.DefaultRequestHeaders.Add("api-key", aoaiKey);
            var data = new
            {
                prompt = prompt,
                n = 1,
                size = "1024x1024"
            };
    
            var jsonData = JsonSerializer.Serialize(data);
            var contentData = new StringContent(jsonData, Encoding.UTF8, "application/json");
            var response = await client.PostAsync(api, contentData);
    
            // Get the revised prompt and image URL from the response
            var stringResponse = await response.Content.ReadAsStringAsync();
            JsonNode contentNode = JsonNode.Parse(stringResponse)!;
            JsonNode dataCollectionNode = contentNode!["data"];
            JsonNode dataNode = dataCollectionNode[0]!;
            JsonNode revisedPrompt = dataNode!["revised_prompt"];
            JsonNode url = dataNode!["url"];
            Console.WriteLine(revisedPrompt.ToJsonString());
            Console.WriteLine(url.ToJsonString().Replace(@"\u0026", "&"));
    
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    } Complete Code on GitHub Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!! AzureOpenAI/samples/Azure.OpenAI.ImageGenerationExamples at main · ssukhpinder/AzureOpenAI C# Programming🚀 Thank you for being a part of the C# community! Before you leave: If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️ Follow us: | | | | | X LinkedIn Dev.to Hashnode Newsletter Tumblr Visit our other platforms: | | | | GitHub Instagram Tiktok Quora Daily.dev Also published here

Taking the Azure Open AI Challenge: Image Generation - Day 2

In nearly all of my recent explorations of Generative AI, I've come to realize how important prompts are. That hasn't necessarily translated me into writing better prompts all the time, but being aware of the problem is at least one step towards correcting it, right? One of the things that I thought would help me personally would be to have a tool to quickly compare and contrast different prompts. Obviously, I could just open up multiple tabs in a browser, but I really wanted something more direct and simplified for my purposes. With that in mind, let me share what I've built. Prompt Tester Prompt Tester is my incredibly imaginatively named tool to... um... test prompts. It begins with one prompt: The idea is that you enter your initial prompt, and then click one or more times to add more prompts. When ready, you can then just click "Generate Results": Before showing some example results, let me talk a bit about how it was built. The Code The backend is an simple Node.js server that handles throwing up the static HTML and responding to the front-end API requests. Once again, I thank Todd Sharp for sharing his initial code with me. Here's the boring simple handling portion: incredibly async function handler(req, res) {
	console.log('Entered handler.', req.method, req.url);

	if(req.method === 'GET' && req.url.indexOf('favicon.ico') === -1) {
		res.writeHead(200, { 'Content-Type':'text/html' });
		res.write(fs.readFileSync('./demo.html'));
		res.end();
	} else if(req.method === 'POST' && req.url === '/api') {

		let body = '';
		req.on('data', chunk => {
			body += chunk.toString();
		});

		req.on('end', async () => {
			body = JSON.parse(body);
			console.log('BODY:\n', JSON.stringify(body, null, '\t'));

			let result = await callGemini(body.prompt);
			res.writeHead(200, { 'Content-Type':'application/json' });
			res.write(JSON.stringify(result));
			res.end();

		});

	}

}

const server = http.createServer(handler);
server.listen(3000);
console.log('Listening on port 3000'); And the part which just passes the prompt off to Google's . In theory, this could be changed to ChatGPT or any other service. callGemini Gemini API const MODEL_NAME = "gemini-pro";
const API_KEY = process.env.GOOGLE_AI_KEY;

const genAI = new GoogleGenerativeAI(API_KEY);
const model = genAI.getGenerativeModel({ model: MODEL_NAME });

async function callGemini(text) {

	const generationConfig = {
		temperature: 0.9,
		topK: 1,
		topP: 1,
		maxOutputTokens: 2048,
	};

	const safetySettings = [
		{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	];

	const parts = [
    	{text},
  	];

	const result = await model.generateContent({
		contents: [{ role: "user", parts }],
		generationConfig,
		safetySettings,
	});

	try {

		if(result.response.promptFeedback && result.response.promptFeedback.blockReason) {

			return { error: `Blocked for ${result.response.promptFeedback.blockReason}` };
		}
		const response = result.response.candidates[0].content.parts[0].text;
		return { response };
	} catch(e) {
		// better handling
		return {
			error:e.message
		}
	}
	
} I'm proud to say that I'm now, finally, doing some basic error handling and trying to expose to the user (me) why a prompt was blocked. The front end is relatively simple. I used for the core CSS (again, thank you Todd) and some vanilla JavaScript to handle adding new prompts and making the calls. Here's the HTML (minus the inline CSS): Milligram <h1>Prompt Tester</h1>

<p>
This tool helps you see the result of multiple prompts at the same time. The idea is to use similar prompts with different phrasing 
to see how it impacts the result.
</p>

<div id="prompts">
</div>

<template id="promptRow">
<div class="row">
	<div class="column column-33"><textarea placeholder="Enter your prompt."></textarea></div>
	<div class="column column-67 result"></div>
</div>
<hr>
</template>


<div class="float-right">
<button id="addPrompt">Add Prompt</button> <button id="generateResults">Generate Results</button>
</div> And then the JavaScript. Notice I used above to handle the prompt layout. This is the first time I've used that tag, and it was relatively easy. <template> let $template, $prompts, $addPromptsBtn, $generateResultsBtn;
let promptCount = 0;

document.addEventListener('DOMContentLoaded', init, false);
async function init() {
	
	$addPromptsBtn = document.querySelector('#addPrompt');
	$addPromptsBtn.addEventListener('click',  addPrompt, false);
	$generateResultsBtn = document.querySelector('#generateResults')
	$generateResultsBtn.addEventListener('click',  generateResults, false);
	$template = document.querySelector('#promptRow');
	$prompts = document.querySelector('#prompts');

	addPrompt();
}

function addPrompt() {
	promptCount++;
	const newPrompt = $template.content.cloneNode(true);
	$prompts.append(newPrompt);
}

async function generateResults() {
	// disable add prompt and generate
	$addPromptsBtn.setAttribute('disabled', 'disabled');
	$generateResultsBtn.setAttribute('disabled', 'disabled');

	let $results = document.querySelectorAll('div.result');
	// clear any previous results
	$results.forEach(r => r.innerHTML = '');

	/*
	gather prompts from the inputs
	*/
	let prompts = [];
	let $fields = document.querySelectorAll('#prompts textarea');
	for(f of $fields) prompts.push(f.value);

	let promises = [];
	for(p of prompts) promises.push(getResult(p));
	let results = await Promise.all(promises);

	// assign results to result area for each, handle errors - later
	// there's a one to one correspondance between our array of results and the result div
	for(let i=0;i<results.length;i++) {
		if(results[i].response) $results[i].innerHTML = marked.parse(results[i].response);
		else if(results[i].error) $results[i].innerHTML = `<strong>Error with Prompt: </strong> ${results[i].error}`;
	}

	$addPromptsBtn.removeAttribute('disabled');
	$generateResultsBtn.removeAttribute('disabled');

}

async function getResult(p) {
	console.log('get result for',p);
	// quick return on empty prompt
	if(p.trim() === '') return { response:'<strong>Enter a prompt.</strong>' };
	let req = await fetch('/api', { method:'POST', body: JSON.stringify({prompt:p.trim()})});
	let result = await req.json();
	return result;
} I that's all pretty simple DOM manipulation and API calling, but definitely let me know in a comment below if something doesn't make sense. Notice I fire off all my API calls at once and wait for them to finish. think Promise.all Also, note I use the library. The results from Google typically have Markdown, and this helps render them nicely in my little web app. marked All of this isn't terribly interesting I think, but let's look at some testing. As I said, I the style of prompt was important. Having it presented directly to me in a tool went a of a long way to reinforcing that. knew heck Sample Result - Gravity Let's start with a simple idea - explaining gravity but for different age groups. Because of the large size of the results, I'm going to use a bit of CSS here to help keep things sane. Prompt: explain gravity to me but make it appropriate for a child in 1st grade Gravity is a superpower that our planet has! It's like an invisible force that makes everything on Earth stay on the ground. Imagine you have a ball in your hand. When you drop it, what happens? It falls down, right? That's because of gravity. Gravity pulls the ball towards the center of the Earth, and so it falls to the ground. Gravity is also what keeps us on the ground. It's what keeps our feet planted firmly on the Earth and prevents us from floating away! Gravity is a very important force. It keeps everything in place and helps things work the way they do. Without gravity, we would all be floating around in space, and things would be very chaotic! Here's a fun way to think about gravity: imagine that the Earth is a giant magnet, and everything on Earth is made of metal. The magnet pulls the metal objects towards it, and that's what keeps them on the ground. Gravity is a very important and interesting force. It's one of the things that makes our planet such a unique and special place to live! Prompt: explain gravity to me but make it appropriate for a child in 7th grade Imagine you're playing with a ball. You toss it up in the air, and it comes back down to you. This is because of gravity. Gravity is a force that pulls things toward each other. It's what keeps you on the ground and what makes things fall down. The amount of gravity between two objects depends on their mass. The more mass an object has, the stronger its gravity is. That's why things like planets and stars have more gravity than things like people and cars. Gravity is also what keeps the planets in orbit around the sun. The sun's gravity is much stronger than the Earth's, so it pulls the Earth towards it. This keeps the Earth moving in a circle around the sun. Gravity is a very important force in our universe. It keeps things in place and makes things move. Without gravity, the planets would fly off into space, and we would all float away. Here are some fun facts about gravity: Gravity is responsible for the tides. The moon's gravity pulls on Earth's oceans, causing them to rise and fall. Gravity is the reason why we have an atmosphere. Without gravity, the air would just float away into space. Gravity is what makes it possible for us to walk. When we walk, our feet push against the ground, and the ground pushes back. This is what propels us forward. Gravity is the reason why we don't float away into space. Earth's gravity pulls us towards its center, keeping us on the ground. Note the typo in the prompt below. Gemini handled it just fine. Prompt: explain gravity to me but make it appropriate for a college-aged student Gravity: The Force of Attraction Gravity is a fundamental force in the universe responsible for keeping celestial bodies, such as planets, moons, and stars, in motion. It is also what keeps us firmly planted to the ground and allows objects to fall when dropped. Newton's Law of Universal Gravitation: The law of universal gravitation states that every object in the universe attracts every other object with a force proportional to the product of their masses and inversely proportional to the square of the distance between them. Mathematically, it can be expressed as: F = Gm1m2/r^2 Where: F is the gravitational force between two objects. G is the gravitational constant (6.674 × 10^-11 N m^2/kg^2). m1 and m2 are the masses of the two objects. r is the distance between the centers of the two objects. Significance of Gravity: The gravitational pull of larger celestial bodies keeps smaller ones revolving around them. For example, Earth's gravity keeps the moon in orbit, while Sun's gravity keeps Earth and other planets in their respective orbits. Keeps Celestial Bodies in Orbit: Gravity between the Earth and the moon causes ocean tides. As the moon orbits Earth, its gravitational pull creates a bulge in the ocean's water on the side facing the moon. Another bulge forms on the opposite side of Earth due to the inertial forces. As the Earth rotates, these bulges move across the planet's surface, resulting in high and low tides. Responsible for Tides: Gravity pulls objects towards Earth's center, preventing them from floating away. This phenomenon allows us to walk, stand, and interact with our environment. Keeps Objects on Earth: Gravity plays a crucial role in the formation of stars and galaxies. The gravitational collapse of massive clouds of gas and dust leads to the creation of new stars. Similarly, the gravitational pull of galaxies keeps them bound together, preventing them from dispersing. Formation of Stars and Galaxies: Albert Einstein's general relativity theory provides a more comprehensive understanding of gravity. It explains that gravity is not a force but rather a curvature of spacetime caused by the presence of mass or energy. This curvature affects the motion of objects, causing them to follow curved paths rather than straight lines. Relativity Theory: In conclusion, gravity is a fundamental force responsible for keeping celestial bodies in motion, tides, and the behavior of objects on Earth. It is a complex phenomenon that has been studied extensively over the centuries, and it continues to fascinate scientists and researchers to this day. Sample Result - Pets For my second test, I started with a basic prompt and then a better version that asked for specific types of details: (Quick editorial note - the tabular data below is a bit messy, but was messy in the original output.) not Prompt: why are cats so much better than dogs? Cats and dogs are both popular pets, and there is no definitive answer to the question of which one is better. Both cats and dogs have their own unique set of advantages and disadvantages, and the best pet for you will depend on your individual needs and preferences. Some people prefer cats because they are generally more independent than dogs. Cats are typically happy to entertain themselves, and they don't need to be walked or taken outside for exercise. This can be a big advantage for people who live in apartments or who have busy schedules. Cats are also known for being clean and tidy animals. They groom themselves regularly, and they typically use the litter box without any problems. This can be a big advantage for people who don't want to deal with the mess and smell of a dog. Of course, there are also some disadvantages to having a cat. Cats can be more expensive than dogs, and they can be more difficult to train. Cats are also more likely to scratch furniture and other household items. Ultimately, the decision of whether to get a cat or a dog is a personal one. There is no right or wrong answer, and the best pet for you will depend on your individual needs and preferences. Here is a table summarizing the key differences between cats and dogs: Characteristic Cat Dog Independence More independent Less independent Exercise needs Does not need to be walked or taken outside Needs to be walked or taken outside Cleanliness Cleans itself regularly Needs to be bathed regularly Training Can be difficult to train Easier to train Scratching May scratch furniture and other household items Does not typically scratch furniture Cost More expensive Less expensive Prompt: why are cats so much better than dogs? explain with a bulleted list of reasons that cover emotional well being and financial aspects Emotional well-being: Independence: Cats are more independent than dogs, which can be a good thing for people who want a pet that doesn't require a lot of attention. Low maintenance: Cats are generally lower maintenance than dogs. They don't need to be walked or groomed as often, and they can be left alone for longer periods of time. Quiet: Cats are generally quieter than dogs, which can be ideal for people who live in apartments or have noise-sensitive neighbors. Therapeutic: Studies have shown that cats can have a calming effect on humans, and they can help to reduce stress and anxiety. Financial aspects: Lower cost: Cats are generally less expensive to own than dogs. They eat less food, they need fewer vet visits, and they don't require as much grooming. Easier to travel with: Cats are easier to travel with than dogs. They can be transported in a carrier, and they don't need to be walked or exercised while you're away. Longer lifespan: Cats have a longer lifespan than dogs, on average. This means that you can enjoy your cat's companionship for many years to come. Sample Result - Chain of Thought Prompting The following test came from an presentation from AWS re:Invent: . I found a lot of great advice from this session, including a good overview of Chain of Thought prompting, where the idea is you provide information about the steps used to solve a problem to help the GenAI engine better solve a prompt. excellent AWS re:Invent 2023 - Prompt engineering best practices for LLMs on Amazon Bedrock (AIM377) Now, the example given in that presentation went like so: Q: Roger has 5 tennis balls. he buys 2 more cans of tennis balls. Each 
can has 3 tennis balls. How many tennis balls does he have now?

A: The answer is 11.

Q. The cafeteria had 23 apples. If they used 20 to make lunch and 
bought 6 more, how many apples do they have? In the presentation, the model in use incorrectly returned 27. In my test, it worked correctly, but I absolutely saw issues with math back when I built my last November. generative AI blackjack advice demo As I said, Gemini handled it fine: But in Prompt Tester, I tried the suggested CoT prompt: A: The answer is 9. Q: Roger has 5 tennis balls. he buys 2 more cans of tennis balls. Each 
can has 3 tennis balls. How many tennis balls does he have now?

A: Roger started with 5 balls. 2 cans of 3 tennis balls each is 6 
tennis balls. 5 + 6 = 11. The answer is 11.

Q. The cafeteria had 23 apples. If they used 20 to make lunch and 
bought 6 more, how many apples do they have? And the result was improved, showing how it got to the answer: A: The cafeteria had 23 apples. They used 20 apples for lunch, which 
means they have 3 apples left. They then bought 6 more apples, so 
they now have a total of 3 + 6 = 9 apples.

The answer is 9. The difference is pretty significant. The Code If this seems useful to you at all, feel free to grab the code from here: . As I mentioned, it shouldn't be that difficult to add or some such and use a different service. If you do, let me know! https://github.com/cfjedimaster/ai-testingzone/tree/main/prompt_tester callChatGPT

How to Test Multiple Variations of Generative AI Prompts

জেনারেটিভ এআই প্রম্পটগুলির একাধিক বৈচিত্র কীভাবে পরীক্ষা করবেন

So testen Sie mehrere Variationen generativer KI-Eingabeaufforderungen

Cómo probar múltiples variaciones de indicaciones de IA generativa

Comment tester plusieurs variantes d'invites d'IA générative

जेनरेटिव एआई प्रॉम्प्ट के एकाधिक विविधताओं का परीक्षण कैसे करें

生成 AI プロンプトの複数のバリエーションをテストする方法

Generative AI 프롬프트의 다양한 변형을 테스트하는 방법

Como testar múltiplas variações de prompts de IA generativos

Как протестировать несколько вариантов подсказок генеративного ИИ

Üretken Yapay Zeka İstemlerinin Çoklu Varyasyonlarını Test Etme

Cách kiểm tra nhiều biến thể của lời nhắc AI sáng tạo

如何测试生成式 AI 提示的多种变体

Way back in January (remember January), I wrote a blog post describing how to use . This worked by uploading the image to Google Gemini, asking for a short description, and using that description for a new filename. Recently, I was thinking about that demo and was curious how well it would work for video games. gen ai to improve image filenames As always, I did a few quick tests in . I did some quick Googling for various games and screenshots, and the results were pretty impressive. Here are three mostly modern examples: Google AI Studio And here's a first failure, identifying this as Final Fantasy 14, not 16. It did well for one old game, although, to be fair, the name is in the picture: really Although failed on this rather obscure one. It's Bard's Tale 2, not Betrayal at Krondor. Also, note it didn't follow my directions to answer with just the name of the game. I then tried a few race games. I was really curious about this as, with the high fidelity of modern racing games, it feels like it would be a difficult task. Surprisingly, it got them right: I then went old school and obscure, and it failed, but it was worth a shot. (The first person to identify it using reverse image search earns 200 Nerd Points.) super without So, all in all... it worked reasonably well. So, let's talk automation! Automating the Process Nearly two years ago, I blogged about to Dropbox. This made use of the fact that the Switch can post to Twitter, and you can use to scrape media from a Twitter account. As far as I know, that probably doesn't work since Twitter crapped the bed on developer access. While it work, the images I got had random names for images. copying Nintendo Switch screenshots Pipedream did The XBox can upload to OneDrive and actually does name the images, which is super helpful. In case you can't read it, the filename is . It's not only got the game in the screenshot but the date and time as well. For the purposes of this blog post, I'm going to ignore that, but in a real application, I'd probably just handle the XBox images with a bit of custom code, so no AI is needed. Diablo IV-2024_02_19-1514-24.png PlayStation is a bit more wonky and a bit surprising. As far as I can tell, you can share videos on YouTube, but screenshots can only be added to the PS app or copied to a USB drive. Again, surprising. Here's a screenshot showing how the images are named, and as you can see, nothing relevant is included. For the purpose of this, let's make some assumptions (nothing ever goes wrong with that, right?). We will assume that our screenshots all get stored in a Dropbox folder named, . SSIn I'm going to build a Pipedream workflow that will: Trigger on a new file added to SSIn. Download the image. Use Gemini AI to determine the game. Use the name as a new folder in Dropbox, under SSOut, So for example, /SSOut/Galaga Upload the image and auto-rename it based on the time. This lets us know when the screenshot was - well, not taken, but at least added to Dropbox. Let's break it down: Step One - The Trigger This part is relatively simple in Pipedream. I created a trigger for new files based on Dropbox action. I specified the path and told it to include a link so it could be downloaded. Step Two - Download the File The next step is another built-in Pipedream action, downloading a file. In this case, it gets downloaded to with the original filename. /tmp Step Three - Use AI to Determine the Game The next step is a Node.js step that essentially takes the code output from AI Studio and has it work with the file in : /tmp import fs from 'fs';
import {
  GoogleGenerativeAI,
  HarmCategory,
  HarmBlockThreshold,
} from "@google/generative-ai";

async function identifyPic(path, key) {
  const MODEL_NAME = "gemini-1.0-pro-vision-latest";
  const API_KEY = key;

  const genAI = new GoogleGenerativeAI(API_KEY);
  const model = genAI.getGenerativeModel({ model: MODEL_NAME });

  const generationConfig = {
    temperature: 0.4,
    topK: 32,
    topP: 1,
    maxOutputTokens: 4096,
  };

  const safetySettings = [
    {
      category: HarmCategory.HARM_CATEGORY_HARASSMENT,
      threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    },
    {
      category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,
      threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    },
    {
      category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
      threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    },
    {
      category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
      threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    },
  ];

  const parts = [
    {text: "Can you identify what game this screenshot comes from? Just tell me the game.\n\n"},
    {
      inlineData: {
        mimeType: "image/png",
        data: Buffer.from(fs.readFileSync(path)).toString("base64")
      }
    },
  ];

  const result = await model.generateContent({
    contents: [{ role: "user", parts }],
    generationConfig,
    safetySettings,
  });

  return result.response.candidates[0].content.parts[0].text;
}

export default defineComponent({
  async run({ steps, $ }) {
    return await identifyPic(steps.download_file_to_tmp.$return_value[1], process.env.PALM_KEY);
  },
}) The only really important part here is the first argument to , the path. Note that the path is the result value from the download step. This definitely surprised me. identifyPic second Step Four - Create a Folder The next step was also a 'built-in,'’ creating the folder. Now, I should warn you. I'm pretty sure this step is going to throw an error if you run twice in the same game. The action's configuration lets you pick a new name if it already exists but doesn't have a "don't make it if it already exists" parameter. I didn't get a chance to test that, but if it an issue, I'd simply switch to the Pipedream feature that lets you hit any Dropbox API with the right credentials, check the Dropbox API docs, and see if it's possible there, or heck, just wrap in a . is try/catch Outside of that, note that I added a to the result from Google as it had a space in front. trim() Also note that sometimes Gemini will return a sentence, not just a game name. For example, "The game is X". I figure that will be pretty obvious in the output and a human can handle that. Step Five - Upload The final step is to upload the picture to the new output directory. To get the name to be date-based, I used a pretty long expression: Did it work? Sure did! I mean it's brittle as heck, but here's the output from a few runs. First, top-level folder: And here's the contents of the Valhalla one: The Code, and Everything Else All in all, I think this works "OK" to "Well", but it is not perfect. I definitely think a human could help, and in fact, one thing Pipedream makes easy is sending emails. You could easily add an email notification at the end where a person could see if something got misfiled. Anyway, this was fun, and if you want to use this yourself in Pipedream, you can find the workflow here: https://github.com/cfjedimaster/General-Pipedream-AI-Stuff/tree/production/identify-and-sort-screenshots-p_brCmQz3 Also published . here

Organizing Video Game Screenshots Using Generative AI

So far, most of my playing with Generative AI has been on the prompt side, with no real look at the various settings you can tweak in your calls. As I'm still very new to this, I'm trying my best to take things slowly. But every time I open and see the settings on the right, I keep nagging myself to take a deeper look. Google AI Studio You can also see the various settings options in the default code Studio spits out: const generationConfig = {
	temperature: 0.9,
	topK: 1,
	topP: 1,
	maxOutputTokens: 2048,
};

const safetySettings = [
	{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
]; I thought today I'd take a look at , as I had a rough idea of what it implied. Temperature, at a high level, refers to how 'varied' a particular generated response can be. As an example, if I were to say, "I like ____", then you could make a list of what the next word would be, rated by how likely it is: temperature very Cats (VERY LIKELY) Star Wars (LIKELY) Books (LIKELY) Vegetables (UNLIKELY) At the highest temperature value, there will be a of variety. At the lowest level, no variety. Now, from what I know, that does mean that every response will be the same. Just very similar. This post, , does a great job explaining temperature (and other settings). lot not "Creatively Deterministic: What are Temperature and Top_P in Generative AI?" While it's relatively easy in AI Studio to adjust the temperature setting you see above, what I really wanted was a tool to help me see multiple results at once. So, I built one. :) Temperature Testing Tool My tool begins with a simple prompt: This is pretty vanilla HTML, so I won't share the code here (I've got a link to everything down below). Upon hitting the button, that's where the magic begins. Here's the JavaScript: // Testing ranges for temperature
const tempRanges = [0, 0.3, 0.6, 1];

// How many times to test per temp.
const perTemp = 2;

let $status, $results;

document.addEventListener('DOMContentLoaded', init, false);
async function init() {
	document.querySelector('#submitBtn').addEventListener('click', handleSubmit, false);
	$status = document.querySelector('#status i');
	$results = document.querySelector('#results');
} I begin with some variable declarations and a listener for . The first two variable sections are the crucial ones. Temperature values go from 0 (focus on only the highest probable response) to 1 (be more creative), and I decided to test four values in that range. DOMContentLoaded For each range in the array of temperatures, I do two tests. My thinking was that two results would be enough to see the variance in one particular temperature value. Carrying on... async function handleSubmit(e) {
	e.preventDefault();
	$results.innerHTML = '';

	let prompt = document.querySelector('#prompt').value.trim();
	if(prompt === '') return;

	/*
	We are going to loop perTemp times for each tempRange. In order to not
	get stopped by Gemini for too many requests, and to provide feedback to the user, 
	I'll report after each tempRange
	*/
	for(temp of tempRanges) {
		$status.innerText = `Getting results for temperature ${temp}.`;
		let promises = [];
		for(let i=0; i<perTemp; i++) {
			promises.push(getResult(prompt, temp));
		}

		let results = await Promise.all(promises);
		let html = `<h2>Temperature ${temp}</h2>`;
		for(let i=0; i < results.length; i++) {
			html += `
<p>
Result #${i+1}:<br/>
${results[i]}
</p>`;
		}
		$results.innerHTML += html;
		console.log(results);
	}

	$status.innerText = ``;
	
}

async function getResult(prompt, temperature) {
	console.log(`Calling getResult with prompt ${prompt} and ${temperature}`);
	let body = {
		prompt, 
		temperature
	};
	let req = await fetch('/api', { method: 'POST', body: JSON.stringify(body)});
	return await req.json();
} In this code, I loop over the range of temperatures and then make my two sample calls. When done, I render it out to the screen. The last function simply passes my prompt and temperature to the backend API. I'll share a screenshot, but it's a wall of text. This is only a small portion: long The backend is a lightweight Node.js server that basically handles outputting my HTML and responding to the request by the client-side JavaScript. Let's take a look at that, specifically the portion handling the Google Gemini call: POST import { GoogleGenerativeAI, HarmCategory, HarmBlockThreshold } from '@google/generative-ai';

const MODEL_NAME = "gemini-pro";
const API_KEY = process.env.GOOGLE_AI_KEY;

const genAI = new GoogleGenerativeAI(API_KEY);
const model = genAI.getGenerativeModel({ model: MODEL_NAME });

async function callGemini(text, temperature) {

	const generationConfig = {
		temperature,
		topK: 1,
		topP: 1,
		maxOutputTokens: 2048,
	};

	const safetySettings = [
		{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	];

	const parts = [
    	{text},
  	];

	const result = await model.generateContent({
		contents: [{ role: "user", parts }],
		generationConfig,
		safetySettings,
	});

	const response = result.response.candidates[0].content.parts[0].text;
	
	return response;
} This is pretty much the exact code spit out by Studio but wrapped into a function where the prompt and temperature can be passed in. This function has error handling, and that absolutely bit me in the butt. I'm going to be taking a look at those errors, and how to handle them, in a future blog post. no So what did I find? Results Here are the results of using the prompt: "Tell me a story about cats." First, temperature 0. Result 1 Once upon a time, in a quaint little town nestled amidst rolling hills, there lived a mischievous cat named Mittens. Mittens was known throughout the neighborhood for her playful antics and her insatiable curiosity. One sunny afternoon, as Mittens was basking in the warmth of the sunbeam that streamed through the window, she noticed a peculiar sound coming from the kitchen. Intrigued, she hopped off the window sill and tiptoed towards the source of the noise. Upon entering the kitchen, Mittens discovered a group of kittens huddled around a bowl of milk. Their tiny paws were eagerly reaching for the sweet liquid, and their eyes sparkled with delight. Mittens, being the adventurous cat she was, couldn't resist joining in on the fun. With a playful leap, Mittens landed gracefully in the midst of the kittens. She purred loudly, her tail swishing back and forth as she rubbed against their soft fur. The kittens, initially startled by the sudden intrusion, soon warmed up to Mittens' friendly presence. Together, they formed a circle around the bowl of milk, their whiskers twitching as they lapped up the delicious treat. Mittens, being the eldest and most experienced cat, took it upon herself to teach the kittens proper table manners. She showed them how to take turns and how to avoid spilling the milk. As the kittens learned and grew, they developed a deep bond with Mittens. They looked up to her as their mentor and protector. Mittens, in turn, cherished her role as the leader of the pack. She spent her days playing with the kittens, teaching them valuable life lessons and keeping them safe from harm. Word of Mittens' extraordinary kindness and wisdom spread throughout the town. People would often bring their cats to her, seeking her advice and guidance. Mittens welcomed each and every cat with open paws, offering them comfort, support, and a sense of belonging. And so, Mittens became known as the "Guardian Cat" of the town. She was loved and respected by all, both humans and cats alike. Her legacy lived on for generations, inspiring countless cats to follow in her paw prints and spread joy and kindness wherever they went. Result 2 Once upon a time, in a quaint little town nestled amidst rolling hills, there lived a mischievous cat named Mittens. Mittens was known throughout the neighborhood for her playful antics and her insatiable curiosity. One sunny afternoon, as Mittens was basking in the warmth of the sunbeam that streamed through the window, she noticed a peculiar sound coming from the kitchen. Intrigued, she hopped off the window sill and tiptoed towards the source of the noise. Upon entering the kitchen, Mittens discovered a group of kittens huddled around a bowl of milk. Their tiny paws were eagerly lapping up the delicious liquid, their whiskers quivering with delight. Mittens, unable to resist the temptation, joined the kittens and began to partake in the feast. As they drank, Mittens couldn't help but notice that one of the kittens was smaller and weaker than the others. It struggled to keep up with its siblings, and its eyes held a look of longing. Mittens, being the compassionate cat that she was, decided to help the little kitten. With a gentle nudge, Mittens pushed the other kittens aside and made room for the weaker one. She then curled her tail around the kitten, providing it with warmth and comfort. The little kitten, grateful for Mittens' kindness, nuzzled against her fur and began to purr softly. As the kittens continued to drink their milk, Mittens couldn't help but feel a sense of fulfillment. She had not only satisfied her own hunger but had also brought joy and comfort to a fellow creature. From that day forward, Mittens became known as the neighborhood's guardian angel. She would often be seen playing with the kittens, teaching them how to climb trees and chase butterflies. She would also bring them food and shelter when they were in need. The kittens, in turn, adored Mittens and would follow her everywhere she went. They would purr and rub against her legs, showing their affection and gratitude. And so, Mittens lived a long and happy life, surrounded by the love and companionship of her feline friends. She taught the town's residents the importance of kindness and compassion, and her legacy lived on long after she was gone. Notice that both start off the same and then diverge. To save on space, I'll skip to temperature 1 next. Result 1 Once upon a time, in a bustling city called Willow Brook, there lived a calico cat named Mittens. Mittens had a reputation for being a fearless explorer, always curious and eager for adventure. One day, Mittens embarked on her biggest expedition yet. She had heard rumors of a secret garden hidden deep within the city park, and she was determined to find it. When nightfall came, Mittens began her adventure, carefully sneaking out of her home and into the park. The park was silent, illuminated only by the soft glow of the moon. As she tiptoed through the winding paths, a faint scent of flowers carried by the wind caught her attention. Mittens followed the scent until she stumbled upon a large tree. Its branches were laden with bountiful blooms, and a gentle breeze sent petals raining down on her. Suddenly, a soft voice whispered, "Welcome, little traveler." Mittens looked around, her eyes widening in astonishment. Sitting on a branch was a beautiful cat; its fur shimmered like the night sky. "I am Esmerelda," the cat said, "and you are Mittens, the brave calico." Mittens was amazed. She had heard stories about Esmerelda, a magical cat said to hold the secret to unlocking the hidden garden. "Yes, Mittens," Esmerelda continued, "you have found the Enchanted Garden. But to enter, you must pass three challenges." Without hesitation, Mittens agreed. Her first challenge was to find a golden key hidden within the roots of an oak tree. With her sharp claws and determined spirit, Mittens discovered the key, its surface glowing brightly. Next, Esmerelda asked Mittens to gather dewdrops from the heart of a moonflower blossom. As Mittens gracefully leaped and climbed the tall stems, she carefully collected the sparkling dew in a tiny seashell. Finally, Mittens' last challenge was to sing a song that would make the trees sway and the flowers bloom. As Mittens lifted her voice, her melodious' tune enchanted the garden, which sprang to life with vibrant colors and dazzling lights. Esmeralda smiled, "Well done, little traveler. You have passed the challenges." With a gentle paw, Esmerelda touched the large oak tree, causing it to slowly open like a door, revealing an awe-inspiring paradise. Inside the Enchanted Garden, Mittens found lush meadows, sparkling fountains, and exotic creatures playing among the trees. It was a place of wonder and joy as if every storybook she had ever heard had come true. Mittens spent the entire night exploring the magical realm, amazed by its beauty. As dawn broke, she turned to Esmerelda and said, " Thank you for sharing this incredible place with me. Esmerelda purred softly. "You are welcome, Mittens. As long as you believe in magic, the garden will always be here for you to explore." From that day forward, Mittens became the guardian of the Enchanted Garden, a place where she could always find solace, friendship, and a sense of belonging among the natural beauty of Willow Brook. And so, the legend of the brave calico and the Enchanting Garden lived on, brought to life of the belief in magic and the wonders that can be found when one dares to explore beyond the ordinary. Result 2 Once upon a time, in a small and charming town called Willow Creek, nestled amidst rolling hills and blooming meadows, there lived two extraordinary cats. One was a sleek and graceful Siamese named Luna, whose shimmering white fur and piercing blue eyes radiated elegance and mystique. Her every movement exuded nimbleness, gliding through the streets of Willow Creek with an air of regal composure. Meet the second cat, a tabby by the name of Max, a playful and mischievous soul adorned with a coat of orange and black stripes. His sparkling green eyes twinkled with a mischievous glint; ears always perked, ever alert to the adventures that awaited him. His playful antics had become a legend among the townspeople, who were often greeted by his friendly antics and joyful meows. The companionship between Luna and Max was a sight to behold. They were inseparable, roaming the cobblestone streets together, their playful pursuits leading them to alleyways and secret paths known only to them. Their friendship was a true testament to the unwavering spirit and unbreakable bond between cats. During the day, they would bask lazily in the warm sunlight that bathed the town square, absorbing the melody of children playing and the laughter of friends sharing stories. At night, under the velvet sky adorned with shimmering stars, they would chase elusive fireflies and embark on daring expeditions, exploring the town's hidden nooks and crannies, their soft paw steps barely disturbing the tranquil silence. Many a local came to rely on the presence of Luna and Max, knowing that their playful chases and affectionate purrs brought an unexplainable joy and warmth to the heart. The elderly, feeling isolated, found solace in their presence, witnessing a simple yet profound beauty in their interactions. Children were captivated by their antics, watching in delight as they climbed trees or chased each other through the flowerbeds. One day, a traveling circus arrived in Willow Creek, filled with fanfare and vibrant colors. Luna and Max, captivated by the excitement, couldn't resist exploring the wonders this circus held. As they ventured into the largest tent, they were greeted with the mesmerising spectacle of trained cats, performing incredible feats of agility and balance. Inspired by their newfound passion, Luna and Max made a daring decision - they ran away to join the circus! They honed their skills tirelessly, practicing their balancing acts and performing stunning leaps. Their dedication and unwavering friendship caught the attention of the ringmaster, who recognized their talent and potential. Finally, the moment arrived when Luna and Max took center stage, their hearts pounding with excitement and anticipation. The audience, holding their breath in anticipation, witnessed an extraordinary performance, a mesmerizing ballet of agility and grace. Luna's elegant poise and Max's playful energy combined to create a symphony of motion that left the spectators awe-inspired. News of Luna and Max's incredible talents spread like wildfire, and they became stars of the circus, sharing their joy and laughter with audiences of all ages. Their unwavering friendship remained the heart of their performances, a symbol of the magical bond between cats that touched the hearts of all who witnessed their amazing journey. And so, Luna and Max, once ordinary cats roaming the streets of Willow Creek, found their destined path in the spotlight of the circus ring, leaving behind a legacy of a friendship that defied the ordinary and inspired the extraordinary in others. As you can tell, even if you only skimmed it, these are much more varied. I found having all these variations on screen at once, from the same prompt, a easy way to help me understand the impact of temperature. If you want to try this yourself, you can grab the bits here: real https://github.com/cfjedimaster/ai-testingzone/tree/main/temp_tester Download the code, the required bits, and set an environment variable for your Google API key. Run the script ( ) and it will open up a web server on port 3000. Lastly, just open that up with your browser. npm i node script.js As always, let me know what you think. I've got some ideas for other tools like this as a way of learning generative AI! Also published . here

Testing Generative AI Temperature Settings with Some Cat Stories

ASP.NET Core is an open-source framework for building modern web applications. NUnit, on the other hand, is a unit-testing framework for C# developers. In this article, I’ll explain how you can use NUnit in testing to ensure that you have coverage on your web applications! ASP.NET Core Testing is a crucial step in software development as it ensures the reliability and quality of code while reducing debugging time and cost. This article touches on the benefits of testing with NUnit and will provide a step-by-step guide on how to implement unit testing in ASP.NET Core to improve your code quality. Let’s dive in! Understanding ASP.NET Core Testing ASP.NET Core is a widely used open-source framework for building web applications, and like any software, it can have issues that need to be addressed. One way to ensure that your application is reliable and meets the user’s needs is to perform rigorous testing. In this section, I will explain the basics of ASP.NET Core testing, the benefits of using NUnit for testing, and how to create an NUnit project to test an ASP.NET Core application. What is Testing in ASP.NET Core? ASP.NET Core testing is the process by which developers test an ASP.NET Core application to ensure that it works as intended. Testing can help to determine issues that may exist in both the front-end and back-end code. ASP.NET Core testing can be done using various testing frameworks, but in this article, we will focus on NUnit. It’s important to acknowledge that we have multiple different testing types available to us, regardless of what framework we’re using. With that said, many people talk about “unit” tests but there are “functional” or “integration” tests as well. I treat “unit” tests as tests that are much more isolated and require mocking out dependencies. In contrast, I use “functional” or “integration” tests to cover much more broad areas of a system. What’s the “best” way to test? The answer, in my opinion, is a mix across the spectrum of what’s available. Prioritize the styles of testing that give you the best levels of confidence in your code. Check out this video on testing approaches for more of my perspective on this: https://youtu.be/iBkSSuQBJx0?embedable=true Benefits of Testing with NUnit NUnit is an open-source unit testing framework for Microsoft .NET that is widely used in the development community. Some benefits of testing with NUnit include: Improved code quality – Automated testing with NUnit ensures that code is working as intended and helps catch errors earlier in the development process. Simplified debugging – By testing with NUnit, developers can quickly identify and isolate issues, which makes debugging much more manageable. Consistent functionality – Running NUnit tests assures developers that changes to the code do not impact the existing functionality of the application. Creating an ASP.NET Core Project for Testing To start unit testing in ASP.NET Core applications with NUnit, you need to have a project to test. You can either create a new project explicitly for testing or use an existing project. A good practice is to keep your test project separate from your production project—this way, any changes you make will only affect the test project. To explain this more clearly – you want to separate what you are using for supporting your tests from the code that you want to ship to production. In the same way that when you download an app to your phone you’re not getting all of the testing code from the app developer, when you deploy a web application you don’t want to deploy all of the test code with it. By separating out your deployable product code from your test code, you have the flexibility to have a dedicated test environment separate from your production environment! Setting up an NUnit Project Setting up an NUnit project is relatively simple and straightforward—there are just a few things to keep in mind. You need to make sure you have the correct NuGet packages installed, and you need to create a class file specifically for your unit tests. Once you’ve set up your NUnit project, you can start writing tests for your application. Let’s make sure you get the packages you need: NUnit – This is the testing framework itself. You’ll need this to be able to mark tests and use assertion logic in your tests! Microsoft.NET.Test.Sdk – If you’re running tests from Visual Studio, you’ll need to ensure you have the .NET test SDK included as well. Without this and JUST having NUnit, you’ll be able to mark your tests but they won’t get picked up to be able to run! NUnit3TestAdapter – Finally, you’ll need this test adapter to run the NUnit tests from inside the Visual Studio test explorer. With these Nuget packages added to your test project, you’re on your way to creating reliable and high-quality software! Next up, we’ll focus on creating the tests themselves. Writing Basic NUnit Tests in CSharp Now that we have a basic understanding of ASP.NET Core testing and how to set up an NUnit project, let’s dive into writing NUnit tests for an ASP.NET Core application. Writing Your First NUnit Test Before we get into the web side of things, the first step in writing NUnit tests is to write a simple test that validates that a method returns the expected output. Let’s say we have a method called on a class called that adds two numbers together. To write a test for this method, we would write the following code: Add Calculator using NUnit.Framework;

[TestFixture]
public class CalculatorTests
{
    [Test]
    public void Add_TwoNumbers_ReturnsSum()
    {
        // Arrange
        Calculator calculator = new Calculator();
        int x = 2;
        int y = 3;

        // Act
        int sum = calculator.Add(x, y);

        // Assert
        Assert.AreEqual(5, sum);
    }
} This code defines a test class, which contains the individual tests. The test method is decorated with the attribute, which tells NUnit that this method is a test. The method has three parts: [Test] Add_TwoNumbers_ReturnsSum Arrange section initializes the variables Act section performs the operation we want to test Assert section compares the actual result to the expected result This is a common format to follow for readable tests. And at this point, we’ve seen the basic setup for marking a test method and defining how we can assert conditions! Using Assertions to Validate Test Results Assertions are a fundamental part of NUnit tests — and all testing frameworks, for that matter. They provide a way for tests to perform comparisons between expected and actual values. The assertion part of a test should draw your attention to what’s truly being validated, and not just exercised. Remember, it’s one thing for a line of code to be executed, but it’s another thing for us to validate that the line of code we executed is doing what we expect! There are several types of assertions available in NUnit, including: – Tests if two values are equal. Assert.AreEqual(expected, actual) – Tests if a condition is true. Assert.IsTrue(condition) – Tests if a condition is false. Assert.IsFalse(condition) – Tests if an object is null. Assert.IsNull(object) – Tests if an object is not null. Assert.IsNotNull(object) These assertions can be used in the Assert section of a test method, as shown in the previous example. Testing Web APIs with NUnit in ASP.NET Core Testing web APIs is an essential part of ensuring that the functionality of an ASP.NET Core application works as intended. To write tests for web APIs, we can use the class provided by the namespace in C#. Here is an example of how to test a basic GET request using NUnit: HttpClient System.Net.Http using NUnit.Framework;
using System.Net.Http;
using System.Threading.Tasks;

[TestFixture]
public class MyApiTests
{
    [Test]
    public async Task Get_EndpointReturnsSuccessAndCorrectContentType()
    {
        // Arrange
        HttpClient client = new HttpClient();
        string url = "https://example.com/api/values";

        // Act
        HttpResponseMessage response = await client.GetAsync(url);
        string responseContent = await response.Content.ReadAsStringAsync();

        // Assert
        response.EnsureSuccessStatusCode();
        Assert.AreEqual("application/json; charset=utf-8",
            response.Content.Headers.ContentType.ToString());
    }
} In this example, we use an object to execute the GET request. The test checks if the response is successful and has the correct content type. HttpClient What’s the big problem with this though? It’s hitting a URL out on the Internet! Even if you change it to localhost or 127.0.0.1, you still need to find a way to manage the lifecycle of the application that you want to test! WebApplicationFactory with NUnit in ASP.NET Core To solve this, we can use a ! I recommend you as an additional primer to spinning up a web app to test. is agnostic to the testing framework you choose (xUNit, NUnit, or whatever else) so you can use it with what you have seen in this article so far. WebApplicationFactory check out this article on using WebApplicationFactory WebApplicationFactory To jump into a concrete code example, let’s see the following: using System.Net.Http;
using System.Threading.Tasks;

using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;

using NUnit.Framework;

[TestFixture]
public class MyApiTests
{
    [Test]
    public async Task Get_EndpointReturnsSuccessAndCorrectContentType()
    {
        // Arrange
        var factory = new WebApplicationFactory<Program>().WithWebHostBuilder(builder =>
        {
            builder.UseEnvironment("Testing");
        });
        var client = factory.CreateClient();
        string url = "https://example.com/api/values";

        // Act
        HttpResponseMessage response = await client.GetAsync(url);
        string responseContent = await response.Content.ReadAsStringAsync();

        // Assert
        response.EnsureSuccessStatusCode();
        Assert.AreEqual("application/json; charset=utf-8",
            response.Content.Headers.ContentType.ToString());
    }
} Please note that the type parameter included here in the is because of using minimal APIs in a web project that we want to test. And if you’re using the minimal API approach, you may need to put this at the end of your program.cs file in your web application to publicly expose your class to your test project, or look into to expose this internal class to your test project: <Program> WebApplicationFactory Program InternalsVisibleTo https://youtu.be/TjsM3oVg5_w?embedable=true Benefits of NUnit in ASP.NET Core Tests Unit testing with NUnit in ASP.NET Core can provide a range of benefits that help create high-quality software. In this section, we will discuss how using NUnit for testing can improve your code’s reliability and quality, as well as speed up the debugging process. Increased Code Reliability and Quality NUnit tests are automated, which means that they can run quickly and easily. Running tests regularly can help you catch errors and spot issues early on in the development lifecycle, allowing for prompt and effective solutions. By running tests regularly with NUnit, teams can be confident that their code is working as it should, and it provides relevant data about the quality of the code. Automatic testing can also ensure that there are no regression bugs that would otherwise go unnoticed. Faster and Easier Debugging Debugging software can be a challenging task for many developers. By running NUnit tests, you can isolate bugs early on in the development cycle and fix them as quickly as possible. NUnit makes use of the “Arrange-Act-Assert” pattern, which allows developers to identify the problem if a test fails. This pattern is particularly useful in allowing developers to isolate errors and produce effective debug reports. Ensuring Consistent Functionality Through Code Changes The functionality of an application can change when code is added or modified. Testing with Nunit in ASP.NET Core allows you to run tests regularly and ensure that any changes to the code don’t impact existing functionality. This process is necessary to ensure consistent functionality across the development lifecycle of any project — and using functional/integration test coverage on ASP.NET Core web APIs can be incredibly powerful for helping build this confidence. NUnit testing can provide a wide range of benefits to developers looking to create high-quality software. By relying on NUnit test-driven development, you can: Improve the reliability and quality of your code Reduce the time and effort needed to debug Help ensure consistent functionality throughout the development cycle Wrapping Up NUnit in ASP.NET Core Learning how to use NUnit in ASP.NET Core is a valuable skill for any C# developer creating web APIs. Not only does it increase the reliability and quality of your code, but it also ensures consistent functionality through code changes. Additionally, debugging becomes faster and easier, saving time and resources. Catching issues now is *WAY* better than hearing from your customer. By leveraging NUnit in ASP.NET Core, you can catch bugs early on, preventing them from becoming larger issues down the line. As a result, you and your development team can produce higher-quality software in less time. I highly recommend incorporating NUnit testing into your work with ASP.NET Core — and ! if not NUnit then at least consider xUnit With a willingness to experiment with different testing approaches, your team can take its testing capabilities to the next level! Mix and match your types of testing coverage to greatly increase your level of confidence in your changes. If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube Also published here.

Getting Started With NUnit in ASP.NET Core

CONTENTWARE

contentware.com

-- employees

Since n.d.

COMPANY RANKING

CONTENTWARE

EVERGREEN INDEX #2839

Representing this company? Click here to claim and customize this page!

Contentware WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

CONTENTWARE

contentware.com

-- employees

Since n.d.

COMPANY RANKING

CONTENTWARE

EVERGREEN INDEX #2839

RELATED COMPANIESRANK

STORIES ABOUT CONTENTWARE

Representing this company? Click here to claim and customize this page!

Contentware WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!