CreatorPad

creatorpad

An increasing number of artists, singers, and authors have entered the blockchain and cryptocurrency markets in recent years. The drive behind the shift is the want to retain ownership of their creations, get just compensation for their efforts, and connect with their audiences on a more personal level. NonFungible that the value of the NFT industry will expand from $41 million in 2018 to $338 million in 2020, with most of this expansion attributable to new entrants into the blockchain and cryptocurrency markets as developers. Non-fungible tokens (NFTs) make it possible for artists and musicians to sell their one-of-a-kind digital creations directly to consumers on blockchain networks. Since galleries and record labels typically take a sizable cut of the profits, this eliminates the need for them. predicts Sharing in profits is just one perk of being involved in the blockchain and cryptocurrency markets. The creators may use the technology to have a more personal connection with their audience. Musicians, for instance, may use blockchain-based platforms to provide fans unique access to backstage material or early album releases, while authors might use NFTs to sell signed copies of their works. In addition, blockchain technology may aid in the defence of authors' rights to their own works. By keeping their digital assets on distributed ledgers, artists may establish their rights to those assets and the legitimacy of their work, making it harder for others to steal their creations. Considering how simple it is to duplicate and distribute digital media, this is of paramount importance. Although getting involved in the blockchain and cryptocurrency market has many advantages, doing so is not without its difficulties. It may be difficult for creators to understand the technical language and elaborate procedures that are part of the blockchain and cryptocurrency industries. Snapmuse.io bridges the gap between web2 and web3 audiences Polygon's new platform, , aims to enable artists earn money from their work and give back to their most devoted followers. The system allows artists to incentivize their communities and provide genuine value to their viewers. Snapmuse.io's combination of NFTs, blockchain technology, and a user-friendly interface streamlines the process of purchasing virtual items. In return, fans purchase CreatorPasses, which provide them three months of access to the artists' private channels and a variety of perks available nowhere else. Snapmuse.io : Vested Interest Disclosure The author is an independent contributor publishing via our . Be it through direct compensation, media partnerships, or networking, the author has a vested interest in the company/ies mentioned in this story. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR brand-as-author program To commemorate the occasion, Mariana Avila, a popular YouTuber, is the first creator to sell her CreatorPass on . On April 29th, Mariana began a campaign offering a free NFT to everyone who signed up for her Snapmuse.io fan club. With 9.5 million YouTube subscribers and 16 million on TikTok, the influencer has a sizable social media following in LATAM. After registering for Snapmuse.io, anyone can claim Mariana's gift. This is a special opportunity for Mariana's fans to attend the online debut of her new music video on May 4th, with the artist herself. Snapmuse.io bridges the gap between web2 and web3 audiences, allowing artists to easily earn steady income while encouraging and rewarding dedicated fan participation. The platform provides artists with a new method of monetization while also assisting them in securing the financial resources necessary to expand their channels and foster deeper, more meaningful connections with their audiences. Snapmuse.io makes it easy to get started with the web3 by requiring just a Google login for access and accepting both credit cards and debit cards for payments. Snapmuse.io Snapmuse.io's readily available artists and the simplicity with which they may join the site might attract millions of new users to web3. The launching of Snapmuse.io on Polygon and the expanding use of blockchain and cryptocurrency in the creative industry suggest that more artists will follow in Mariana Avila's footsteps and join the blockchain and cryptocurrency market. Don’t forget to like and share the story! Image credits: AI.

Snapmuse.io on Polygon: Empowering Artists to Earn and Engage with Fans

Snapmuse.io en Polygon: Capacitar a los artistas para ganar e interactuar con los fanáticos

Snapmuse.io sur Polygon : donner aux artistes les moyens de gagner de l'argent et de s'engager auprès des fans

बहुभुज पर Snapmuse.io: कलाकारों को कमाने और प्रशंसकों के साथ जुड़ने के लिए सशक्त बनाना

Snapmuse.io on Polygon: アーティストが収益を上げてファンと交流できるようにする

Snapmuse.io no Polygon: capacitando artistas para ganhar e se envolver com os fãs

Snapmuse.io trên Polygon: Trao quyền cho các nghệ sĩ kiếm tiền và tương tác với người hâm mộ

Polygon 上的 Snapmuse.io：使艺术家能够赚钱并与粉丝互动

By: Blockchain Developer Jesse Abramowitz Ethereum’s latest last fork has successfully been integrated into Ethereum and with that comes extra features. This article will cover what the optcode is, why it was added, and what you can do with it. Lastly, I will go through an in-depth demonstration of how to execute it. Constantinople create2 Let’s get started! What is Create2? This EIP (Ethereum Improvement Proposal) was created by Vitalik on 2018–04–20 and can be found . Currently there are two types of addresses in Ethereum: wallet addresses and contact addresses. All of these addresses already exist and can have funds sent to them already (there is no button or switch or message required to initiate these account). To create a wallet, a randomly generated private key is required, which will map it to the corresponding public key, then address on chain. here When creating a contract address with the regular create optcode, the address is determined from the address and the nonce of the sender. It would be very dangerous to pre interact with a contract because, (1) the nonce can get out of order and all the funds would be lost and (2) the address is not tied to the contract code so any contract can take that address. This is what the Create2 optcode can fix. Any contract that is deployed with it can have a deterministic address that is calculated by: keccak256( 0xff ++ address ++ salt ++ keccak256(init_code))[12:] Since everything there is pre-known a user can calculate it yourself off chain then send the parameters on chain and create the contract. Why is this important? With this method users can interact with a contract that does not yet exist. Their funds will be safe at that address and when they are ready, they can deploy a contract that fills that address and interact with those funds. This solution is secure and (unlike the Create optcode) allows for binding of that address to the initiating contract code. Use cases The big use cases for this optcode started in the state channel scalability domain. Specifically with pre-creating and funding contracts before they exist. The whole idea of state channels are to reduce cost. If people can interact with a contract before it exists, then when they are ready to exit the state channel they can create, settle and destroy the contract all in one call, that would significantly reduce the cost. Anyways, Counterfactual is cool and if you want to read more about it you can start . here What I always find so great about the crypto and the Ethereum community is how they can turn a slit into a whole. Weird metaphor, but I mean it with utmost admiration. It has been less than a month and people have started theorizing using this optcode for other cool use cases. For example, James Young from his medium post “ ” describes how to onboard new users with Create2 in a more seamless way: One Weird Trick to Fix User On-Boarding Example Steps for Ethereum New User Flow (ENUF) User generated key pair locally (keys never leave the device) Key pair used to generate contract address Funds sent to address to contract deployment prior Contract deployed user has funds to at least pay for gas after For novice users, centralize key recovery (ex. ) Chaperone Wallet Concept Let’s Calculate it Ok, enough talk, let’s see this in action. I found some sample code . And after some alterations, I was able to deploy it to the rinkeby network . You can see a transaction which is my factory contract deploying the Create2 contract. here here My final outcome looked like this: We see the factory contract that I deployed first, then we pre-calculate my new contract, we check to make sure it is not on chain (just for our knowledge) we can see false. Then we run the transaction which calls the factory contract’s create2 optcode. That gives us the transaction hash and from that we get the contract address. Now we can check to see if the contract exists on chain and we can see it is true. The function we are most interested in here it this: function buildCreate2Address(creatorAddress, saltHex, bytecode) {return `0x${web3.utils.sha3(`0x${[‘ff’,creatorAddress,saltHex,web3.utils.sha3(bytecode)].map(x => x.replace(/0x/, ‘’)).join(‘’)}`).slice(-40)}`.toLowerCase()} It takes the creator of the contract’s address, a hex of the salt (any random number of certain length) and the init code to create the contract and then it hashes them. This is the same math the EVM does and thus produces the same contract code. Possible Attack Vectors Since the generation of the contract address relies on the initcode it is not tied to the constructor which gets called in the contract code at run time. A great explanation of that can be found . here This means that there is a possible attack where user’s agree on a contract, pre interact with it. Yet this contract has a suicide code in it, and after it is killed, it is redeployed with different inputs for the constructor. Thus you are re-deploying different code than was thought to be expected. Which can lead to potential poor behaviour. An example of this was launched in this Reddit . thread Conclusion Create2 is an awesome improvement I can’t wait to see what people build with it. Even more, I can’t wait until I can suggest it to a client as the best possible solution and go build something with it myself. — - Jesse Abramowitz is a Blockchain Developer at BlockX Labs. He has worked on multiple DApps, projects, and Blockchain Networks. Currently, he is also a professor at George Brown College in Toronto. He is always looking to help, teach and build on the blockchain. You can reach him at: jesse@blockxlabs.com specializes in building developer tools and solutions for blockchain ecosystems.We aim to sift through the noise to bring some sense and clarity into the Blockchain space. Our accomplishments include — a wallet and DApp interaction tool for the Aion Network, and — a test token faucet for Ethereum, Aion, and Tron. BlockX Labs AIWA Universal Faucet Follow Us on Twitter: @BlockXLabs

Create2: A Tale of Two Optcodes

Dynamic Generation of Airflow DAGs for Multiple Environments

File uploads are a common feature in many web applications, but handling them efficiently and securely can be challenging. In this article, we will explore how to implement efficient file uploads using Node.js, Express.js, and MongoDB. We will cover the entire process, from setting up the server to storing files in MongoDB GridFS, ensuring that your application can handle large files without sacrificing performance. Problem Statement Handling file uploads efficiently involves several challenges: Scalability: Managing large files without overwhelming server memory.
Security: Preventing unauthorized access and ensuring data integrity.
Performance: Uploading and retrieving files quickly.
Storage Management: Efficiently storing and retrieving files from the database. Solution We will use the following stack to address these challenges: Node.js: For the server-side environment.
Express.js: As the web framework.
Multer: Middleware for handling multipart/form-data, which is used for uploading files.
MongoDB: For storing metadata and files.
GridFS: A specification for storing and retrieving large files in MongoDB. Step-by-Step Implementation 1. Setting Up the Project First, let’s create a new Node.js project and install the necessary dependencies. mkdir file-upload-app
cd file-upload-app
npm init -y
npm install express multer mongoose gridfs-stream 2. Creating the Server Next, we will set up a basic Express server. // server.js
const express = require('express');
const mongoose = require('mongoose');
const multer = require('multer');
const { GridFsStorage } = require('multer-gridfs-storage');
const Grid = require('gridfs-stream');
const methodOverride = require('method-override');
const bodyParser = require('body-parser');
const crypto = require('crypto');
const path = require('path');

const app = express();

// Middleware
app.use(bodyParser.json());
app.use(methodOverride('_method'));
app.set('view engine', 'ejs');

// Mongo URI
const mongoURI = 'mongodb://localhost:27017/fileupload';

// Create mongo connection
const conn = mongoose.createConnection(mongoURI, { useNewUrlParser: true, useUnifiedTopology: true });

// Init gfs
let gfs;
conn.once('open', () => {
  // Init stream
  gfs = Grid(conn.db, mongoose.mongo);
  gfs.collection('uploads');
});

// Create storage engine
const storage = new GridFsStorage({
  url: mongoURI,
  file: (req, file) => {
    return new Promise((resolve, reject) => {
      crypto.randomBytes(16, (err, buf) => {
        if (err) {
          return reject(err);
        }
        const filename = buf.toString('hex') + path.extname(file.originalname);
        const fileInfo = {
          filename: filename,
          bucketName: 'uploads'
        };
        resolve(fileInfo);
      });
    });
  }
});
const upload = multer({ storage });

// @route POST /upload
// @desc  Uploads file to DB
app.post('/upload', upload.single('file'), (req, res) => {
  res.json({ file: req.file });
});

// @route GET /files
// @desc  Display all files in JSON
app.get('/files', (req, res) => {
  gfs.files.find().toArray((err, files) => {
    // Check if files
    if (!files || files.length === 0) {
      return res.status(404).json({
        err: 'No files exist'
      });
    }

    // Files exist
    return res.json(files);
  });
});

// @route GET /files/:filename
// @desc  Display single file object
app.get('/files/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }
    return res.json(file);
  });
});

// @route GET /image/:filename
// @desc Display Image
app.get('/image/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }

    // Check if image
    if (file.contentType === 'image/jpeg' || file.contentType === 'image/png') {
      // Read output to browser
      const readstream = gfs.createReadStream(file.filename);
      readstream.pipe(res);
    } else {
      res.status(404).json({
        err: 'Not an image'
      });
    }
  });
});

// @route DELETE /files/:id
// @desc  Delete file
app.delete('/files/:id', (req, res) => {
  gfs.remove({ _id: req.params.id, root: 'uploads' }, (err, gridStore) => {
    if (err) {
      return res.status(404).json({ err: err });
    }
    res.redirect('/');
  });
});

const port = 5000;

app.listen(port, () => console.log(`Server started on port ${port}`)); 3. Testing the Upload Feature We can test our implementation using tools like Postman. Upload a File: Use a POST request to http://localhost:5000/upload with form data containing the file.
Retrieve Files: Use a GET request to http://localhost:5000/files to get the list of all files.
Retrieve a Single File: Use a GET request to http://localhost:5000/files/:filename to get a specific file’s details.
Display Image: Use a GET request to http://localhost:5000/image/:filename to view the image in the browser.
Delete a File: Use a DELETE request to http://localhost:5000/files/:id to delete a file. Conclusion In this article, I demonstrated how to handle file uploads efficiently using Node.js, Express.js, and MongoDB. By leveraging GridFS, we can store and manage large files seamlessly, ensuring scalability, security, and performance. This approach can be further customized and expanded based on specific application requirements, making it a versatile solution for file management in modern web applications. File uploads are a common feature in many web applications, but handling them efficiently and securely can be challenging. In this article, we will explore how to implement efficient file uploads using Node.js, Express.js, and MongoDB. We will cover the entire process, from setting up the server to storing files in MongoDB GridFS, ensuring that your application can handle large files without sacrificing performance. Problem Statement Handling file uploads efficiently involves several challenges: Scalability: Managing large files without overwhelming server memory. Security: Preventing unauthorized access and ensuring data integrity. Performance: Uploading and retrieving files quickly. Storage Management: Efficiently storing and retrieving files from the database. Scalability : Managing large files without overwhelming server memory. Scalability Security : Preventing unauthorized access and ensuring data integrity. Security Performance : Uploading and retrieving files quickly. Performance Storage Management : Efficiently storing and retrieving files from the database. Storage Management Solution We will use the following stack to address these challenges: Node.js: For the server-side environment. Express.js: As the web framework. Multer: Middleware for handling multipart/form-data, which is used for uploading files. MongoDB: For storing metadata and files. GridFS: A specification for storing and retrieving large files in MongoDB. Node.js : For the server-side environment. Node.js Express.js : As the web framework. Express.js Multer : Middleware for handling multipart/form-data , which is used for uploading files. Multer multipart/form-data MongoDB : For storing metadata and files. MongoDB GridFS : A specification for storing and retrieving large files in MongoDB. GridFS Step-by-Step Implementation 1. Setting Up the Project First, let’s create a new Node.js project and install the necessary dependencies. mkdir file-upload-app
cd file-upload-app
npm init -y
npm install express multer mongoose gridfs-stream mkdir file-upload-app
cd file-upload-app
npm init -y
npm install express multer mongoose gridfs-stream 2. Creating the Server Next, we will set up a basic Express server. // server.js
const express = require('express');
const mongoose = require('mongoose');
const multer = require('multer');
const { GridFsStorage } = require('multer-gridfs-storage');
const Grid = require('gridfs-stream');
const methodOverride = require('method-override');
const bodyParser = require('body-parser');
const crypto = require('crypto');
const path = require('path');

const app = express();

// Middleware
app.use(bodyParser.json());
app.use(methodOverride('_method'));
app.set('view engine', 'ejs');

// Mongo URI
const mongoURI = 'mongodb://localhost:27017/fileupload';

// Create mongo connection
const conn = mongoose.createConnection(mongoURI, { useNewUrlParser: true, useUnifiedTopology: true });

// Init gfs
let gfs;
conn.once('open', () => {
  // Init stream
  gfs = Grid(conn.db, mongoose.mongo);
  gfs.collection('uploads');
});

// Create storage engine
const storage = new GridFsStorage({
  url: mongoURI,
  file: (req, file) => {
    return new Promise((resolve, reject) => {
      crypto.randomBytes(16, (err, buf) => {
        if (err) {
          return reject(err);
        }
        const filename = buf.toString('hex') + path.extname(file.originalname);
        const fileInfo = {
          filename: filename,
          bucketName: 'uploads'
        };
        resolve(fileInfo);
      });
    });
  }
});
const upload = multer({ storage });

// @route POST /upload
// @desc  Uploads file to DB
app.post('/upload', upload.single('file'), (req, res) => {
  res.json({ file: req.file });
});

// @route GET /files
// @desc  Display all files in JSON
app.get('/files', (req, res) => {
  gfs.files.find().toArray((err, files) => {
    // Check if files
    if (!files || files.length === 0) {
      return res.status(404).json({
        err: 'No files exist'
      });
    }

    // Files exist
    return res.json(files);
  });
});

// @route GET /files/:filename
// @desc  Display single file object
app.get('/files/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }
    return res.json(file);
  });
});

// @route GET /image/:filename
// @desc Display Image
app.get('/image/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }

    // Check if image
    if (file.contentType === 'image/jpeg' || file.contentType === 'image/png') {
      // Read output to browser
      const readstream = gfs.createReadStream(file.filename);
      readstream.pipe(res);
    } else {
      res.status(404).json({
        err: 'Not an image'
      });
    }
  });
});

// @route DELETE /files/:id
// @desc  Delete file
app.delete('/files/:id', (req, res) => {
  gfs.remove({ _id: req.params.id, root: 'uploads' }, (err, gridStore) => {
    if (err) {
      return res.status(404).json({ err: err });
    }
    res.redirect('/');
  });
});

const port = 5000;

app.listen(port, () => console.log(`Server started on port ${port}`)); // server.js
const express = require('express');
const mongoose = require('mongoose');
const multer = require('multer');
const { GridFsStorage } = require('multer-gridfs-storage');
const Grid = require('gridfs-stream');
const methodOverride = require('method-override');
const bodyParser = require('body-parser');
const crypto = require('crypto');
const path = require('path');

const app = express();

// Middleware
app.use(bodyParser.json());
app.use(methodOverride('_method'));
app.set('view engine', 'ejs');

// Mongo URI
const mongoURI = 'mongodb://localhost:27017/fileupload';

// Create mongo connection
const conn = mongoose.createConnection(mongoURI, { useNewUrlParser: true, useUnifiedTopology: true });

// Init gfs
let gfs;
conn.once('open', () => {
  // Init stream
  gfs = Grid(conn.db, mongoose.mongo);
  gfs.collection('uploads');
});

// Create storage engine
const storage = new GridFsStorage({
  url: mongoURI,
  file: (req, file) => {
    return new Promise((resolve, reject) => {
      crypto.randomBytes(16, (err, buf) => {
        if (err) {
          return reject(err);
        }
        const filename = buf.toString('hex') + path.extname(file.originalname);
        const fileInfo = {
          filename: filename,
          bucketName: 'uploads'
        };
        resolve(fileInfo);
      });
    });
  }
});
const upload = multer({ storage });

// @route POST /upload
// @desc  Uploads file to DB
app.post('/upload', upload.single('file'), (req, res) => {
  res.json({ file: req.file });
});

// @route GET /files
// @desc  Display all files in JSON
app.get('/files', (req, res) => {
  gfs.files.find().toArray((err, files) => {
    // Check if files
    if (!files || files.length === 0) {
      return res.status(404).json({
        err: 'No files exist'
      });
    }

    // Files exist
    return res.json(files);
  });
});

// @route GET /files/:filename
// @desc  Display single file object
app.get('/files/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }
    return res.json(file);
  });
});

// @route GET /image/:filename
// @desc Display Image
app.get('/image/:filename', (req, res) => {
  gfs.files.findOne({ filename: req.params.filename }, (err, file) => {
    if (!file || file.length === 0) {
      return res.status(404).json({
        err: 'No file exists'
      });
    }

    // Check if image
    if (file.contentType === 'image/jpeg' || file.contentType === 'image/png') {
      // Read output to browser
      const readstream = gfs.createReadStream(file.filename);
      readstream.pipe(res);
    } else {
      res.status(404).json({
        err: 'Not an image'
      });
    }
  });
});

// @route DELETE /files/:id
// @desc  Delete file
app.delete('/files/:id', (req, res) => {
  gfs.remove({ _id: req.params.id, root: 'uploads' }, (err, gridStore) => {
    if (err) {
      return res.status(404).json({ err: err });
    }
    res.redirect('/');
  });
});

const port = 5000;

app.listen(port, () => console.log(`Server started on port ${port}`)); 3. Testing the Upload Feature We can test our implementation using tools like Postman. Upload a File: Use a POST request to http://localhost:5000/upload with form data containing the file. Retrieve Files: Use a GET request to http://localhost:5000/files to get the list of all files. Retrieve a Single File: Use a GET request to http://localhost:5000/files/:filename to get a specific file’s details. Display Image: Use a GET request to http://localhost:5000/image/:filename to view the image in the browser. Delete a File: Use a DELETE request to http://localhost:5000/files/:id to delete a file. Upload a File : Use a POST request to http://localhost:5000/upload with form data containing the file. Upload a File http://localhost:5000/upload Retrieve Files : Use a GET request to http://localhost:5000/files to get the list of all files. Retrieve Files http://localhost:5000/files Retrieve a Single File : Use a GET request to http://localhost:5000/files/:filename to get a specific file’s details. Retrieve a Single File http://localhost:5000/files/:filename Display Image : Use a GET request to http://localhost:5000/image/:filename to view the image in the browser. Display Image http://localhost:5000/image/:filename Delete a File : Use a DELETE request to http://localhost:5000/files/:id to delete a file. Delete a File http://localhost:5000/files/:id Conclusion In this article, I demonstrated how to handle file uploads efficiently using Node.js, Express.js, and MongoDB. By leveraging GridFS, we can store and manage large files seamlessly, ensuring scalability, security, and performance. This approach can be further customized and expanded based on specific application requirements, making it a versatile solution for file management in modern web applications.

Efficient File Uploads in Node.js: Using Express, MongoDB, and GridFS for Scalable Storage

This Is How You Can Develop a Decentralized Email System on the Blockchain

I first discovered way back in 2021 when I built a demo of their APIs for the Adobe Developer blog ( ). Diffbot "Natural Language Processing, Adobe PDF Extract, and Deep PDF Intelligence" At that time, I was impressed with how easy Diffbot's API was and also how quickly it responded. I had not looked at their API in a while, but a few days ago, they announced new support for summarizing text. I thought this would be a great thing to combine with the . Here's what I found. Adobe PDF Extract API First off, if you want to try this yourself, you'll need: Adobe PDF Services . These are free, and you get 500 transactions per month for free. For folks who may not know, I work for Adobe, and this is one of the products I cover. credentials Diffbot . They provide a free two-week trial but no free tier. That being said, I've had to reach out to them a few times when building stuff, and they've provided really great support for me, so I definitely think they're worth you checking out. credentials Alright, let's look at how a summary flow might work. Step One - Extract the Text The (sorry, the "Adobe PDF Extract API", wait, this is my blog, I can shorten things!) is pretty powerful. It uses AI to intelligently parse a PDF to correctly find each and every element detail in the document. Extract API So text, fonts, colors, position, and so forth. It can also find images and tabular data as well which leads to some pretty powerful use cases. (For a good example of this, see my where I scan multiple scientific journals to collect and aggregate astronomical data and create reports.) blog post For this demo, we literally just need the text. For that, I'll make use of the REST APIs. The "flow" for nearly all aspects of the PDF services available are: Exchange credentials for an access token Ask to upload a file for input (in this case, a PDF to be extracted) Upload the document Kick off the job Poll for completion Download the bits Note that there are also SDKs you can use, but I've found our REST APIs so simple I just hit the endpoints directly. Here's the script I wrote to do the Extract process. It's basically everything I said above and pointing to a source PDF in my local filesystem. /*
This demo is a two step process. This file, step one, handles extracting and storing the JSON from a PDF.
*/
import 'dotenv/config';
import fs from 'fs';
import { Readable } from 'stream';
import { finished } from 'stream/promises';

const SOURCE_PDF = '../../source_pdfs/boring_adobe_security_doc.pdf';
const REST_API = "https://pdf-services.adobe.io/";

const CLIENT_ID = process.env.CLIENT_ID;
const CLIENT_SECRET = process.env.CLIENT_SECRET;

async function delay(x) {
	return new Promise(resolve => {
		setTimeout(() => resolve(), x);
	});
}

async function getAccessToken(id, secret) {

	const params = new URLSearchParams();
	params.append('client_id', id);
	params.append('client_secret', secret);

	let resp = await fetch('https://pdf-services-ue1.adobe.io/token', { 
		method: 'POST', 
		headers: {
			'Content-Type':'application/x-www-form-urlencoded'
		},
		body:params 
	});

	let data = await resp.json();
	return data.access_token;
}

async function getUploadData(mediaType, token, clientId) {

	let body = {
		'mediaType': mediaType
	};
	body = JSON.stringify(body);

	let req = await fetch(REST_API+'assets', {
		method:'post',
		headers: {
			'X-API-Key':clientId,
			'Authorization':`Bearer ${token}`,
			'Content-Type':'application/json'
		},
		body: body
	});

	let data = await req.json();
	return data;
}

async function uploadFile(url, filePath, mediaType) {

	let stream = fs.createReadStream(filePath);
	let stats = fs.statSync(filePath);
	let fileSizeInBytes = stats.size;

	let upload = await fetch(url, {
		method:'PUT', 
		redirect:'follow',
		headers: {
			'Content-Type':mediaType, 
			'Content-Length':fileSizeInBytes
		},
		duplex:'half',
		body:stream
	});

	if(upload.status === 200) return;
	else {
		throw('Bad result, handle later.');
	}

}

async function pollJob(url, token, clientId) {

	let status = null;
	let asset; 

	while(status !== 'done') {
		let req = await fetch(url, {
			method:'GET',
			headers: {
				'X-API-Key':clientId,
				'Authorization':`Bearer ${token}`,
			}
		});

		let res = await req.json();

		status = res.status;
		if(status === 'done') {
			asset = res;
		} else {
			await delay(2000);
		}
	}

	return asset;
}

async function downloadFile(url, filePath) {
	let res = await fetch(url);
	const body = Readable.fromWeb(res.body);
	const download_write_stream = fs.createWriteStream(filePath);
	return await finished(body.pipe(download_write_stream));
}

async function extractJob(asset, token, clientId) {
	let body = {
		'assetID': asset.assetID
	}

	let resp = await fetch(REST_API + 'operation/extractpdf', {
		method: 'POST', 
		headers: {
			'Authorization':`Bearer ${token}`, 
			'X-API-KEY':clientId,
			'Content-Type':'application/json'
		},
		body:JSON.stringify(body)
	});

	return resp.headers.get('location');

}

let accessToken = await getAccessToken(CLIENT_ID, CLIENT_SECRET);
console.log('Got our access token.');

let uploadedAsset = await getUploadData('application/pdf', accessToken, CLIENT_ID);

await uploadFile(uploadedAsset.uploadUri, SOURCE_PDF, 'application/pdf');
console.log('Source PDF Uploaded.');

let job = await extractJob(uploadedAsset, accessToken, CLIENT_ID);
console.log('Job created. Now to poll it.');

let result = await pollJob(job, accessToken, CLIENT_ID);
console.log('Job is done.'); 

await downloadFile(result.content.downloadUri, 'extract.json');
console.log('All done.'); Ok, hopefully, you're still reading. In general, I try to avoid posting giant blocks of code like that, but if you focus on the lines at the end, you'll see I'm just hitting utility functions that do what I described in the flow above. Authenticate, ask to upload a PDF, kick off a job, check it, and download the result. One note I'll add. Extract returns a zip file containing a JSON result set, and optionally, tables and images. One nice thing about the REST API is that I can get directly to the JSON and just store it. The JSON result can be quite huge. For my source PDF (an incredibly boring Adobe security document) of three pages, the resulting JSON is 4560 lines long. You can find my source PDF , and the raw output from Extract . here here Instead of putting all 4.5k lines here, let me show a snippet - two unique elements found by the API: {
	"Bounds": [
		44.62139892578125,
		756.9429931640625,
		245.0037841796875,
		766.3184967041016
	],
	"Font": {
		"alt_family_name": "* Arial",
		"embedded": true,
		"encoding": "Identity-H",
		"family_name": "* Arial",
		"font_type": "CIDFontType0",
		"italic": false,
		"monospaced": false,
		"name": "*Arial-6539",
		"subset": false,
		"weight": 400
	},
	"HasClip": false,
	"Lang": "en",
	"Page": 0,
	"Path": "//Document/Sect/P",
	"Text": "Adobe Vendor Security Review Program White Paper ",
	"TextSize": 8.5,
	"attributes": {
		"SpaceAfter": 18
	}
},
{
	"Bounds": [
		0.0,
		0.0,
		630.0,
		820.7799987792969
	],
	"ClipBounds": [
		548.72802734375,
		739.1929931640625,
		602.5444488525391,
		820.7799987792969
	],
	"Page": 0,
	"Path": "//Document/Sect/Figure",
	"attributes": {
		"BBox": [
			548.9779999999737,
			739.4429999999993,
			587.61599999998,
			790.920999999973
		],
		"Placement": "Block"
	}
}, In the sample above, you can see the first element is textual and contains a property, while the second one is a figure. For my demo, I just need to use the property when it exists. Let's see that in action. Text Text Step Two - Create the Summary I mentioned earlier that the Diffbot API was fairly simple to use. Let me demonstrate that. First, I'll set up some variables and read in the JSON I got from the first step. To be clear, I could do everything in one process, but there's really no point in running Extract more than once. What's cool is - I could actually do calls on the result. multiple As an example, one other cool feature Diffbot has is to get entities from text, i.e., what a document is speaking about (people, places, etc). Anyway, here's the beginning: /*
In this file, we take the result from our Extract operation and pass it to Diffbot
*/

import 'dotenv/config';
import fs from 'fs';

const DIFFBOT_KEY = process.env.DIFFBOT_KEY;
const SOURCE_JSON = './extract.json';
const data = JSON.parse(fs.readFileSync(SOURCE_JSON, 'utf8'));

console.log(`Read in source data from ${SOURCE_JSON}.`); Next, I need to parse out the text from the Extract result: let text = data.elements.reduce((text, el) => {
	if(el.Text) text += el.Text + '\n';
	return text;
},''); Next, I craft an HTTP request to Diffbot. Check their for more information. docs let fields = 'summary';
let url = `https://nl.diffbot.com/v1/?fields=${fields}&token=${DIFFBOT_KEY}`;
	
let body = [{
	content:text, 
	lang:'en',
	format:'plain text'
}];

console.log('Passing text to Diffbot.'); 

let req = await fetch(url, { 
	method:'POST',
	body:JSON.stringify(body),
	headers: { 'Content-Type':'application/json' }
});

let result = await req.json(); And that's it. As a final step, I simply output it: console.log(`Summary of PDF:\n${result[0].summary}`); Given the source PDF, the final result is: Adobe has a vendor security review program that evaluates vendors that collect, store, process, 
transmit, or dispose of Adobe data outside of Adobe-controlled physical offices or data center 
locations. The VSR program includes requirements for vendors to follow when handling sensitive 
data and assigns a risk level score to vendors based on their compliance with Adobe standards. 
If a vendor fails the VSR program, Adobe holds discussions with the business owner to understand 
the details of the vendor's security practices and determine whether or not to continue working with them. My three-page PDF is now one simple paragraph. You can imagine how useful this would be for organizations with millions of documents. Combine this with other services (like the entities feature I mentioned previously), and it makes working with large libraries that much easier. Try It! If you want to check this out yourself, you can grab all the code here: . As I said, everything here can be tested for free, so give it a shot, and let me know what you think in a comment below. https://github.com/cfjedimaster/document-services-demos/tree/main/random_demos/extract_diffbot_summary

How to Use AI and PDF Services to Automate Document Summaries

নথির সারাংশ স্বয়ংক্রিয় করতে এআই এবং পিডিএফ পরিষেবাগুলি কীভাবে ব্যবহার করবেন

So nutzen Sie KI- und PDF-Dienste zur Automatisierung von Dokumentzusammenfassungen

Cómo utilizar servicios de IA y PDF para automatizar resúmenes de documentos

Comment utiliser les services AI et PDF pour automatiser les résumés de documents

दस्तावेज़ सारांशों को स्वचालित करने के लिए AI और PDF सेवाओं का उपयोग कैसे करें

AI および PDF サービスを使用して文書の要約を自動化する方法

AI 및 PDF 서비스를 사용하여 문서 요약을 자동화하는 방법

Como usar serviços de IA e PDF para automatizar resumos de documentos

Как использовать сервисы AI и PDF для автоматизации сводки документов

Belge Özetlerini Otomatikleştirmek için Yapay Zeka ve PDF Hizmetleri Nasıl Kullanılır?

Cách sử dụng dịch vụ AI và PDF để tự động hóa việc tóm tắt tài liệu

如何使用 AI 和 PDF 服务自动化文档摘要

Machine Learning: Your Ultimate Feature Selection Guide Part 2 - Select the Real Best

Guess what? Microsoft just dropped a new version of like clockwork! Keeping up with bug fixes and API changes can feel like a full-time gig, and missing out on cool features is a snap. But fret not; we've got your back! Playwright Join us on a journey through the latest Playwright updates. We're here to help you stay in the loop, pick up some new tricks, and wow your colleagues with your stellar skills! Ready to dive in? Let's go! Playwright Latest Updates (v1.04) If you're eager to get a firsthand look at the latest Playwright updates through some awesome examples, head over to the and watch the monthly “What's New in Playwright” video: Playwright YouTube Channel https://www.youtube.com/watch?v=mn892dV81_8&embedable=true Time to explore the latest features introduced in Playwright and see how to get the most out of them 🔍 New APIs field added to the object argument of . That method launches a browser using persistent storage located at and returns the browser context instance. `is an object containing the Firefox user preferences as specified at . FirefoxUserPrefs options browserType.launchPersistentContext(userDataDir, options) userDataDir firefoxUserPrefs about:config field added to the object argument of the , , and methods. is a string containing the error message reported by all operations interrupted as a result of a call. reason options page.close(options) browserContext.close(options) browser.close(options) reason close() While caters specifically to Firefox users in need of custom configs, the field is much more general purpose. Use it as in the snippet below: firefoxUserPrefs reason await browser.close({
    reason: "Scraping process completed!"
}) All pending operations interrupted by will now throw a JavaScript error with the message “Scraping process completed!” browser.close() How to use this feature? Imagine your target page is in the midst of a critical task—let's say, waiting for some data to be retrieved. Suddenly, an unexpected error pops up, and you're faced with the need to gracefully close the browser. Without , you'd be left clueless about whether the resources for the ongoing task have been released and why the operation was interrupted. Not knowing what's going on is bad, especially when you're tasked with inspecting the reason for an error in the logs of an automated web scraping script. That's where the `reason` field comes to your rescue, saving you days of painstaking investigation. reason Awesome, this API introduction makes debugging a lot easier! 🚀 New Functionality for the Test Generator Tool The , designed to create tests automatically as you perform actions in a browser, now provides the following buttons: Playwright Test Generator tool : Verify that the selected element is visible by generating a instruction. Assert visibility expect(locator).toBeVisible() : Make sure that the selected HTML element contains specific text via a instruction Assert text expect(locator).toContainText() : Check that the select element has a particular value by adding a instruction to your test. Assert value expect(locator).toHaveValue() : you’re working on a complex task, such as . Assume you need to make sure that some elements on the page are visible and contain specific text or values. That would involve some complex logic 👎. But hold up—thanks to this fresh update, it all boils down to a few clicks in the Test Generator tool! Picture this building a web scraper for a dynamic web page See this new feature in action in the following GIF: That Test Generator interaction will produce the following TypeScript test for you: import { test, expect } from '@playwright/test';
   test('test', async ({ page }) => {
   await page.goto('https://playwright.dev/');
   await expect(page.getByRole('banner')).toContainText('Get started');
   }); ✨ Pretty magical, isn’t it? ✨ Updated Browser Versions In the grand tradition of Playwright's major updates, the lineup of supported browsers has been updated with newer versions: Chromium 120.0.6099.28 Mozilla Firefox 119.0 WebKit 17.4 But that's not all! The current version of Playwright has also proven excellent against the following stable channels: Google Chrome 119 Microsoft Edge 119 Other Minor Changes The and methods now throw an error when the download operation fails or gets canceled. download.path() download.createReadStream() The Playwright docker image now comes with Node.js v20. Don't want to miss out on any new updates? Keep an eye on the Playwright page! Release Notes How to Update Playwright Now, you must be thinking, “How can I get my hands on those fantastic new additions?” Well, by updating Playwright to the latest version, my friend! Just fire up the command below: npm install @playwright/test@latest And forget to upgrade the browser instances with: npx playwright install Voilà! You're all set to get your hands on the latest Playwright updates! New Playwright, Same Old Problems… No matter how up-to-date your version of Playwright is, most sites will still be able to detect and block your automated scripts. But how is that even possible? Well, headless browsers controlled by libraries like Playwright involve special configurations and settings that are seen as red flags by anti-bot solutions. The consequence? Immediate blocks or the unwelcome appearance of CAPTCHA and . other pesky obstacles Now, you might be thinking, “Can't I just tweak my browser settings to avoid this?” Not so fast, kiddo! That’s not a great idea for at least three compelling reasons: It's a never-ending cat-and-mouse game—anti-bot measures evolve, making today's workaround old news by tomorrow. Even with the slickest browser configurations, excessive requests from the same IP might still earn you suspicious glances from the target site. User interactions like form submissions may require CAPTCHA solving, which isn’t a walk in the park to automate! The issue isn’t with the browser automation library itself (Playwright rocks! 🤘), but rather with the browser under control. The solution would be a highly customizable browser that: Runs in headed mode like a normal browser to avoid bot detection. Can easily scale to the cloud to save you time and money in infrastructure management. Provides rotating IPs backed by one of the widest and most reliable proxy networks on the market Can automatically manage CAPTCHA solving, browser fingerprinting, cookie and headers customization, and automatic retries for you for maximum effectiveness. Integrates with the most popular browser automation libraries, such as Playwright, Selenium, and Puppeteer Believe it or not, that isn't some far-off mirage. That's real and is exactly what solution is all about! Bright Data’s Scraping Browser Final Thoughts Playwright is the rock star of browser automation libraries, and just like Santa Claus delivers presents on Christmas Eve, Microsoft releases a major new update every month. Here, you've seen how to get the most out of the latest Playwright updates, but let's face it, they won't magically make you invisible to sites with advanced bot detection technologies. Dodge that bullet with the Scraping Browser solution from and join our mission to make the Internet a public place for everyone, everywhere, even through automated scripts! Bright Data Until next time, keep exploring the Web with freedom!

How to Make the Most of Playwright After the Latest Updates

সর্বশেষ আপডেটের পরে কীভাবে নাট্যকারের সর্বাধিক সুবিধা পাবেন

So holen Sie nach den neuesten Updates das Beste aus Playwright heraus

Cómo aprovechar al máximo Dramaturgo después de las últimas actualizaciones

Comment tirer le meilleur parti de Playwright après les dernières mises à jour

नवीनतम अपडेट के बाद नाटककार का अधिकतम लाभ कैसे उठाएं

Como aproveitar ao máximo o dramaturgo após as últimas atualizações

Как максимально эффективно использовать возможности драматурга после последних обновлений

En Son Güncellemelerden Sonra Oyun Yazarından En İyi Şekilde Nasıl Yararlanılır?

Cách tận dụng tối đa Playwright sau khi cập nhật mới nhất

Is Your Code Slow?: Avoid These 19 Common JavaScript and Node.js Mistakes

This article focuses on using OpenAI's library to build a that trains the Davinci model in mathematics. Node.js CLI Nutshell "Scaffold" our library. Write a set of functions to wrap OpenAI's API calls. Build a simple CLI to call the functions. Prove ChatGPT is (usually) good at math. Prove Davinci is (usually) bad at math. Build a simple fine-tuning dataset for teaching Davinci math. Upload "simple fine-tuning dataset". Turn a "simple fine-tuning dataset" into a simple fine-tuning model. Proving our fine-tuning taught Davinci math. Scaffold cd ~/Dev/YourRootFolderForPersonalStuff/
mdkir davinci-is-bad-at-maths
cd davinci-is-bad-at-maths
npm i dotenv openai
npm i prettier -D
touch .env
touch goodAtMathsDatasetBuilder.js
touch openAI.js
mkdir bin
touch bin/cli.js package.json ... can be simple, like this: {
  "description": "Experiments using OpenAI's API NodeJs v4 library",
  "name": "davinci-is-bad-at-maths",
  "private": true,
  "bin": "./bin/cli.js",
  "dependencies": {
    "dotenv": "^16.3.1",
    "openai": "^4.0.0"
  },
  "devDependencies": {
    "prettier": "^3.0.2"
  },
  "main": "openAI.js",
  "scripts": {
    "cli": "node bin/cli.js",
    "prettier": "prettier --list-different --write \"**/*.{css,html,js,json,md,mjs,scss,ts,yaml}\""
  },
  "type": "module"
} The "cli" entry in scripts means we can call . If you use this instead of it means you maintain your 's history even if you change the app structure later, or the name of . Simple things please simple minds and I have a simple mind. npm run cli -- commandName [args] node bin/cli.js commandName [args] shell cli.js .env ... must look like this but with your own API_KEY: OPENAI_API_KEY="sk-d0ntY0uD4reUs3MyK3yG3tY0urOwnFr0mOp0n41W36s1t3Yo"
OPENAI_MODEL="davinci" A set of functions to wrap OpenAI's API calls. Open and copy this in: openAI.js /** A not-robust OpenAI v4 CLI; a playground for OpenAI v4 API calls; a utility for working with a OpenAI model who is really really, like - I mean - really bad at maths.
 * @usage
 * >> import commandHub from "openAI.js"
 * >> const [, , command, ...args] = process.argv
 * >> const commandFunc = commandHub[command]
 * >> commandFunc(...args)
 */
import fs from "fs"
import dotenv from "dotenv"
import OpenAI from "openai"

dotenv.config()

// Fine Tuning only works with davinci, curie, babbage, and ada, so we will put which in our .env file so that we can call the same one consistently.
const model = process.env.OPENAI_MODEL

// Instantiate the API object.
const apiKey = process.env.OPENAI_API_KEY
const openai = new OpenAI({ apiKey })

/** openai.chat.completions.create
 * @usage
 * >> npm run cli -- chatCompletionsCreate "2+8=?"
 * @param {String} chatPrompt your sum to an assistent who is (usually) good at maths */
export const chatCompletionsCreate = async chatPrompt => {
  const res = await openai.chat.completions.create({
    messages: [
      { role: "system", content: "You are good at maths." },
      { role: "user", content: chatPrompt },
    ],
    model: model,
  })
  console.log("chatCompletionsCreate", res.choices)
}

/** openai.completions.create
 * @tutorial
 * Normally we would use `chatCompletionsCreate` but for Fine Tuned models we must use base models and therefore `completionsCreate`.
 * @usage
 * >> npm run cli -- completionsCreate "2+8=?"
 * @param {String} chatPrompt your sum to an assistent who is (usually) good at maths */
export const completionsCreate = async chatPrompt => {
  const res = await openai.completions.create({
    model: model,
    prompt: chatPrompt,
    temperature: 0,
  })
  console.log("completionsCreate", res)
}

/** openai.files.create and output to `openai.files.create.json`
 * @usage
 * >> npm run cli -- filesCreate bad-at-maths-fine-tuning-dataset.jsonl
 * @param {String} filePath of JSONLD file to upload. */
export const filesCreate = async filePath => {
  const res = await openai.files.create({
    file: fs.createReadStream(filePath),
    purpose: "fine-tune",
  })
  console.log("filesCreate", res)
  fs.writeFileSync(
    "openai.files.create.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}
// openai.files.del

/** openai.files.list and output to `openai.files.list.json`
 * @usage
 * >> npm run cli -- filesList */
export const filesList = async () => {
  const res = await openai.files.list()
  console.log("filesList", res)
  fs.writeFileSync(
    "openai.files.list.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}
// openai.files.retrieve
// openai.files.retrieveContent

/** openai.fineTunes.create
 * @usage
 * >> npm run cli -- fineTunesCreate "bad-at-maths-fine-tuning-dataset.jsonl" "is-good-at-maths"
 * @param {String} fileId of previously uploaded file where `purpose: "fine-tune"`.
 * @param {String} suffix to add to the resulting model name for easily id later.  */
export const fineTunesCreate = async (fileId, suffix) => {
  const res = await openai.fineTunes.create({
    training_file: fileId,
    suffix: suffix,
    model: model,
  })
  console.log("fineTunesCreate", res)
  fs.writeFileSync(
    "openai.fineTunes.create.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}

/** openai.fineTunes.list
 * @usage
 * >> npm run cli -- fineTunesList */
export const fineTunesList = async () => {
  const res = await openai.fineTunes.list()
  console.log("fineTunesList", res)
  fs.writeFileSync(
    "openai.fineTunes.list.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}

// openai.fineTunes.cancel
// openai.fineTunes.retrieve
// openai.fineTunes.listEvents
// openai.models.del
// openai.models.list
// openai.models.del
// openai.images.generate
// openai.images.edit
// openai.images.createVariation
// openai.audio.transcriptions.create
// openai.audio.translations.create
// openai.edits.create
// openai.embeddings.create
// openai.moderations.create

// A command hub.
const commandHub = {
  chatCompletionsCreate,
  completionsCreate,
  filesCreate,
  filesList,
  fineTunesCreate,
  fineTunesList,
}

export default commandHub You'll notice I have left all the available endpoints in 's library in this file, which I leave for you to add as an exercise to create a useful module. OpenAI A simple CLI to call the functions Open bin/cli.js and paste this: #!/usr/bin/env node
/** A not-very-robust OpenAI v4 CLI; a playground for OpenAI v4 API calls; a utility for working with a OpenAI model who is really really, like - I mean - really bad at maths.
 * @usage with "cli" in "scripts" (don't forget the "--").
 * >> npm cli -- commandName [arg1 arg2 ...arg(n)]
 */
import commandHub from "../openAI.js"

const [, , command, ...args] = process.argv

// Call the requested command. Not a robust CLI but it gets the job done!
if (!commandHub.hasOwnProperty(command)) {
  throw "No such command as `" + command + "`"
} else {
  const commandFunc = commandHub[command]
  commandFunc(...args)
} Prove ChatGPT is (usually) good at math ChatGPT should have no problems answering any sums because (usually) ChatGPT is good at maths, which we can prove (and test our CLI) by doing the following: Edit .env to say: OPENAI_API_KEY="sk-d0ntY0uD4reUs3MyK3yG3tY0urOwnFr0mOp0n41W36s1t3Yo"
OPENAI_MODEL="gpt-3.5-turbo" Run the command: npm run cli -- chatCompletionsCreate "12+4`. See? Good at maths. At a later date, when it becomes possible to Fine Tune chatbot models like "gpt-3.5-turbo", we will Fine Tune it to be bad at maths. part is required to ensure the parameters are passed correctly into NPM. I won't go into why because I don't know why. You might. That's good. Let me know if you know. All I know is that you have to do it to make it work and that's a fact. The -- NB: This is how you would do the same thing outside of our CLI: import dotenv from "dotenv"
import OpenAI from "openai"

const apiKey = process.env.OPENAI_API_KEY
const model = process.env.OPENAI_MODEL
const openai = new OpenAI({ apiKey })

const chatCompletionsCreate = async chatPrompt => {
  const res = await openai.chat.completions.create({
    messages: [
      { role: "system", content: "You are good at maths." },
      { role: "user", content: chatPrompt },
    ],
    model: model,
  })
  console.log("chatCompletionsCreate", res.choices)
}

chatCompletionsCreate("12+4") Prove Davinci is (usually) bad at math. Edit .env to say: OPENAI_API_KEY="sk-d0ntY0uD4reUs3MyK3yG3tY0urOwnFr0mOp0n41W36s1t3Yo"
OPENAI_MODEL="davinci" Run the command npm run cli -- completionsCreate "12+4`. NB: This is how you would do the same thing outside of our CLI: import fs from "fs"
import dotenv from "dotenv"
import OpenAI from "openai"

const apiKey = process.env.OPENAI_API_KEY
const openai = new OpenAI({ apiKey })
const completionsCreate = async chatPrompt => {
  const res = await openai.completions.create({
    model: model,
    prompt: chatPrompt,
    temperature: 0,
  })
  console.log("completionsCreate", res)
}

completionsCreate("12+4") Teaching DaVinci maths As per the documentation, "Fine Tuning" ChatGPT of models requires large datasets, at least 200. The whole point of is how to create, upload and use "Fine Tuning" datasets and shortcut the actually BUILDING a useful rather-than-silly dataset. davinci-is-bad-at-maths learn work And since we are coders, we can code a shortcut like this: Open and paste this: goodAtMathsDatasetBuilder.js import fs from "fs"

// Don't waste bandwidth with duplicates in the fine-training data.
const data = new Set()

// Build a list of 500 sums which have been done correctly.
while (data.size < 500) {
  // Two random integers.
  let x = Math.round(Math.random() * 1000)
  let y = Math.round(Math.random() * 1000)
  let result = x + y

  data.add(
    JSON.stringify({
      prompt: `${x}+${y}\n\n###\n\n`,
      completion: `${x}+${y}=${result} END`,
    }),
  )
}

fs.writeFileSync(
  "good-at-maths-fine-tuning-dataset.jsonl",
  [...data].join("\n"),
  "utf-8",
)

console.log("JSONL fine-tuning dataset has been created.") All we're doing here is building a data set that "Fine Tunes" ChatGPT models to be good at maths, and all we need is lots of sums with "completions" which are correct. Run this script like this: node goodAtMathsDatasetBuilder.js` Open and it should look like this: good-at-maths-fine-tuning-dataset.jsonl {"prompt":"487+63\n\n###\n\n","completion":"487+63=550 END"}
{"prompt":"842+624\n\n###\n\n","completion":"842+624=1466 END"}
{"prompt":"58+783\n\n###\n\n","completion":"58+783=841 END"}
{"prompt":"96+478\n\n###\n\n","completion":"96+478=574 END"}
{"prompt":"69+401\n\n###\n\n","completion":"69+401=470 END"} ... with more sums that are right. Upload "simple fine-tuning dataset". To upload the dataset, run npm run cli -- filesCreate good-at-maths-fine-tuning-dataset.jsonl NB: This is how you would do the same thing outside of our CLI: import fs from "fs"
import dotenv from "dotenv"
import OpenAI from "openai"

const apiKey = process.env.OPENAI_API_KEY
const openai = new OpenAI({ apiKey })

const filesCreate = async filePath => {
  const res = await openai.files.create({
    file: fs.createReadStream(filePath),
    purpose: "fine-tune",
  })
  console.log("filesCreate", res)
  fs.writeFileSync(
    "openai.files.create.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}

filesCreate("good-at-maths-fine-tuning-dataset.jsonl") Take note of the file , e.g. "file-th15IsM1ne3G3tY0urOwn1Yo" id Turn a "simple fine-tuning dataset" into a simple fine-tuning model To create a "Fine Tuned" model using this dataset call: npm run cli -- fineTunesCreate "file-th15IsM1ne3G3tY0urOwn1Yo"`"is-good-at-maths" NB: This is how you would do the same thing outside of our CLI: import fs from "fs"
import dotenv from "dotenv"
import OpenAI from "openai"

const apiKey = process.env.OPENAI_API_KEY
const openai = new OpenAI({ apiKey })

const fineTunesCreate = async (fileId, suffix) => {
  const res = await openai.fineTunes.create({
    training_file: fileId,
    suffix: suffix,
    model: model,
  })
  console.log("fineTunesCreate", res)
  fs.writeFileSync(
    "openai.fineTunes.create.json",
    JSON.stringify(res, null, 2),
    "utf-8",
  )
}

fineTunesCreate("file-th15IsM1ne3G3tY0urOwn1Yo") It takes a while to teach Davinci maths because, to be honest, DaVinci is really bad at maths! You can run: npm run cli -- fineTunesList Wait until changes to status: 'pending' status: 'suceeded' Proving our fine tuning taught davinci math When , find the name. status: 'suceeded' fine_tuned_model Edit .env to say: OPENAI_API_KEY="sk-d0ntY0uD4reUs3MyK3yG3tY0urOwnFr0mOp0n41W36s1t3Yo"
OPENAI_MODEL="<fine_tuned_model name>" Run: npm run cli -- completionsCreate "12+4`. It's a hokey response, but you should see that Davinci is better at maths. What we learnt How to use OpenAI's V4 library. How to create a "Fine Tuning" dataset and upload it. How to generate a new OpenAI model. How to write a crappy CLI. This project can be found here: https://gitlab.com/timitee/davinci-is-bad-at-maths/edit#js-general-project-settings

Davinci Is Bad at Maths: Fine-Tuning ChatGPT Models With NodeJs and OpenAI v4

Davinci গণিতে খারাপ: NodeJs এবং OpenAI v4 এর সাথে চ্যাটজিপিটি মডেলের ফাইন-টিউনিং

Davinci ist schlecht in Mathematik: Feinabstimmung von ChatGPT-Modellen mit NodeJs und OpenAI v4

Davinci es malo en matemáticas: ajuste fino de los modelos de ChatGPT con NodeJs y OpenAI v4

Davinci est mauvais en maths : affiner les modèles ChatGPT avec NodeJs et OpenAI v4

डेविंसी गणित में खराब है: NodeJs और OpenAI v4 के साथ ChatGPT मॉडल को ठीक से ट्यून करना

Davinci は数学が苦手です: NodeJs と OpenAI v4 を使用した ChatGPT モデルの微調整

Davinci는 수학에 약합니다: NodeJ 및 OpenAI v4를 사용하여 ChatGPT 모델 미세 조정

Davinci é ruim em matemática: ajustando modelos ChatGPT com NodeJs e OpenAI v4

Davinci Matematikte Kötü: NodeJ'ler ve OpenAI v4 ile ChatGPT Modellerine İnce Ayar Yapma

Davinci kém môn toán: Tinh chỉnh các mô hình ChatGPT với NodeJs và OpenAI v4

达芬奇数学不好：使用 NodeJs 和 OpenAI v4 微调 ChatGPT 模型

Stories, News & Mentions about CREATORPAD

Creatorpad’s Stories on HackerNoon