You'll be working on our collaborative writing and production platform for TV and film

Scripto

scripto

Architectural Designs

Of the power of the eleventh horn of Daniel's fourth Beast, to change times and laws

Welcome to other chapters of Let’s Understand Chrome V8 In the past articles, we talked a lot about the JavaScript compiler of V8. The parser, the scanner, and the bytecode, as well as their fundamentals, kernel code, and key structure, were all we covered. In the next articles, we will walk through the compiler workflow, and watch how V8 transforms the JavaScript code into bytecode step by step. Figure 1 shows the workflow, it starts from the JavaScript code you wrote, goes through the scanner and parser, and finally generates bytecodes. In this article, I use d8.exe instead of v8.exe, since d8.exe can print out to the terminal directly as well as the d8 is very light versus v8. Note: 1. Read JavaScript code Below is our test case： function ignition(s) {
    this.slogan=s;
	this.start=function(){eval('console.log(this.slogan);')}
}
worker = new ignition("here we go!");
worker.start(); Below is the Execute() that is responsible for executing the JavaScript code. Quite simply, it first reads the JavaScript code from a file, then compiles and executes the code. 1.  bool SourceGroup::Execute(Isolate* isolate) {
2.  //............omit..................
3.      // Use all other arguments as names of files to load and run.
4.      HandleScope handle_scope(isolate);
5.      Local<String> file_name =
6.          String::NewFromUtf8(isolate, arg, NewStringType::kNormal)
7.              .ToLocalChecked();
8.      Local<String> source = ReadFile(isolate, arg);
9.      if (source.IsEmpty()) {
10.        printf("Error reading '%s'\n", arg);
11.        base::OS::ExitProcess(1);
12.      }
13.      Shell::set_script_executed();
14.      if (!Shell::ExecuteString(isolate, source, file_name, Shell::kNoPrintResult,
15.                                Shell::kReportExceptions,
16.                                Shell::kProcessMessageQueue)) {
17.        success = false;
18.        break;
19.      }
20.    }
21.    return success;
22.  } In line 5, the NewFromUtf8 gets the file name which is our case. In line 8, the ReadFile gets file content which is our case actually. 1.  Local<String> Shell::ReadFile(Isolate* isolate, const char* name) {
2.  //only the most important parts...............................
3.    char* chars = static_cast<char*>(file->memory());
4.    Local<String> result;
5.    if (i::FLAG_use_external_strings && i::String::IsAscii(chars, size)) {
6.      String::ExternalOneByteStringResource* resource =
7.          new ExternalOwningOneByteStringResource(std::move(file));
8.      result = String::NewExternalOneByte(isolate, resource).ToLocalChecked();
9.    } else {
10.      result = String::NewFromUtf8(isolate, chars, NewStringType::kNormal, size)
11.                   .ToLocalChecked();
12.    }
13.    return result;
14.  } Line 8 and line 10 return the JavaScript code depending on the type is ExternalOneByte or UTF8. Here, our case is UTF8, regarding String::ExternalOneByteStringResource, we will talk about it in the future. Back to SourceGroup::Execute(), and step into Shell::ExecuteString at line 14, its source is given below: 1.  bool Shell::ExecuteString(Isolate* isolate, Local<String> source,
2.                      Local<Value> name, PrintResult print_result,
3.                      ReportExceptions report_exceptions,
4.                      ProcessMessageQueue process_message_queue) {
5.  	//omit............................
6.  bool success = true;
7.  {
8.    if (options.compile_options == ScriptCompiler::kConsumeCodeCache) {
9.  	//omit............................
10.       } else if (options.stress_background_compile) {
11.  	//omit............................
12.       } else {
13.         ScriptCompiler::Source script_source(source, origin);
14.         maybe_script = ScriptCompiler::Compile(context, &script_source,
15.                                                options.compile_options);
16.       }
17.       Local<Script> script;
18.       if (!maybe_script.ToLocal(&script)) {
19.         // Print errors that happened during compilation.
20.         if (report_exceptions) ReportException(isolate, &try_catch);
21.         return false;
22.       }
23.       if (options.code_cache_options ==
24.           ShellOptions::CodeCacheOptions::kProduceCache) {
25.  	//omit............................
26.       }
27.       maybe_result = script->Run(realm);//here is executing bytecode.
28.  }
29.  } In line 13, our JavaScript is wrapped into a variable script_source which includes the line and column offset. Since V8 only compiles the JavaScript function that is executing exactly not the full JavaScript code, the variable script_source help the compiler record the compilation information. In line 14, start to compile JavaScript. 2. Parser initialization In the workflow, the first part is the scanner, the second is the parser. Actually, the scanner is passive and the parser is active, this means that the parser takes out a token from the compilation cache actively, and the scanner is waked up by the parser once the cache miss, and to generate tokens and fill the cache. Below is CompileUnboundInternal() called by ScriptCompiler::Compile that at above line 14. 1.  MaybeLocal<UnboundScript> ScriptCompiler::CompileUnboundInternal(
2.      Isolate* v8_isolate, Source* source, CompileOptions options,
3.      NoCacheReason no_cache_reason) {
4.  //omit...............
5.    i::Handle<i::String> str = Utils::OpenHandle(*(source->source_string));
6.    i::Handle<i::SharedFunctionInfo> result;
7.    i::Compiler::ScriptDetails script_details = GetScriptDetails(
8.        isolate, source->resource_name, source->resource_line_offset,
9.        source->resource_column_offset, source->source_map_url,
10.        source->host_defined_options);
11.    i::MaybeHandle<i::SharedFunctionInfo> maybe_function_info =
12.        i::Compiler::GetSharedFunctionInfoForScript(
13.            isolate, str, script_details, source->resource_options, nullptr,
14.            script_data, options, no_cache_reason, i::NOT_NATIVES_CODE);
15.    if (options == kConsumeCodeCache) {
16.      source->cached_data->rejected = script_data->rejected();
17.    }
18.    delete script_data;
19.    has_pending_exception = !maybe_function_info.ToHandle(&result);
20.    RETURN_ON_FAILED_EXECUTION(UnboundScript);
21.    RETURN_ESCAPED(ToApiHandle<UnboundScript>(result));
22.  } The above function generates the UnboundInternal stuff, the line 11 tells us that the stuff is just a Sharedfunction actually. But, what is Bind? A Sharedfunction cannot be executed directly, V8 needs to match a context to the Sharedfunction, the “match” is just the bing. Let’s step into GetSharedFunctionInfoForScript. 1.  MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
2.      Isolate* isolate, Handle<String> source,
3.      const Compiler::ScriptDetails& script_details,
4.   .................) {
5.  //omit.........................
6.  		{
7.      maybe_result = compilation_cache->LookupScript(
8.          source, script_details.name_obj, script_details.line_offset,
9.          script_details.column_offset, origin_options, isolate->native_context(),
10.          language_mode);
11.    }
12.    if (maybe_result.is_null()) {
13.      ParseInfo parse_info(isolate);
14.      // No cache entry found compile the script.
15.      NewScript(isolate, &parse_info, source, script_details, origin_options,
16.                natives);
17.      // Compile the function and add it to the isolate cache.
18.      if (origin_options.IsModule()) parse_info.set_module();
19.      parse_info.set_extension(extension);
20.      parse_info.set_eager(compile_options == ScriptCompiler::kEagerCompile);
21.      parse_info.set_language_mode(
22.          stricter_language_mode(parse_info.language_mode(), language_mode));
23.      maybe_result = CompileToplevel(&parse_info, isolate, &is_compiled_scope);
24.      Handle<SharedFunctionInfo> result;
25.      if (extension == nullptr && maybe_result.ToHandle(&result)) {
26.        DCHECK(is_compiled_scope.is_compiled());
27.        compilation_cache->PutScript(source, isolate->native_context(),
28.                                     language_mode, result);
29.      } else if (maybe_result.is_null() && natives != EXTENSION_CODE) {
30.        isolate->ReportPendingMessages();
31.      }
32.    }
33.    return maybe_result;
34.  } Line 7 is looking up the compilation cache, that I mentioned in the last article. Line 13, the parse_info is a wrapper for Parser, like the variable script_source mentioned upfront. Line 15, the Parser_info initialization, code is given below: 1.  Handle<Script> NewScript(Isolate* isolate, ParseInfo* parse_info,
2.                           Handle<String> source,
3.                           Compiler::ScriptDetails script_details,
4.                           ScriptOriginOptions origin_options,
5.                           NativesFlag natives) {
6.    Handle<Script> script =
7.        parse_info->CreateScript(isolate, source, origin_options, natives);
8.    Handle<Object> script_name;
9.    if (script_details.name_obj.ToHandle(&script_name)) {
10.      script->set_name(*script_name);
11.      script->set_line_offset(script_details.line_offset);
12.      script->set_column_offset(script_details.column_offset);
13.    }
14.    Handle<Object> source_map_url;
15.    if (script_details.source_map_url.ToHandle(&source_map_url)) {
16.      script->set_source_mapping_url(*source_map_url);
17.    }
18.    Handle<FixedArray> host_defined_options;
19.    if (script_details.host_defined_options.ToHandle(&host_defined_options)) {
20.      script->set_host_defined_options(*host_defined_options);
21.    }
22.    return script;
23.  } From line 6 to line 12 wrap the JavaScript code that is our case into the variable script, then initialize the line_offset and column_offset. It is like the script_source really as I said. Back to Compiler::GetSharedFunctionInfoForScript(), and step into CompileToplevel() at line 23. 1.  MaybeHandle<SharedFunctionInfo> CompileToplevel(
2.      ParseInfo* parse_info, Isolate* isolate,
3.      IsCompiledScope* is_compiled_scope) {
4.  //omit.......................
5.    if (parse_info->literal() == nullptr &&
6.        !parsing::ParseProgram(parse_info, isolate)) {
7.      return MaybeHandle<SharedFunctionInfo>();
8.    }
9.  //omit........................
10.    MaybeHandle<SharedFunctionInfo> shared_info =
11.        GenerateUnoptimizedCodeForToplevel(
12.            isolate, parse_info, isolate->allocator(), is_compiled_scope);
13.    if (shared_info.is_null()) {
14.      FailWithPendingException(isolate, parse_info,
15.                               Compiler::ClearExceptionFlag::KEEP_EXCEPTION);
16.      return MaybeHandle<SharedFunctionInfo>();
17.    }
18.    FinalizeScriptCompilation(isolate, parse_info);
19.    return shared_info;
20.  } In line 5, the literal() returns the abstract syntax tree(AST), and V8 starts up the compiler to generate AST if is nullptr. The first time, the AST is null, and steps into ParseProgram. 1.  bool ParseProgram(ParseInfo* info, Isolate* isolate,
2.                    ReportErrorsAndStatisticsMode mode) {
3.  //omit............................
4.    Parser parser(info);
5.    FunctionLiteral* result = nullptr;
6.    result = parser.ParseProgram(isolate, info);
7.    info->set_literal(result);
8.    if (result) {
9.      info->set_language_mode(info->literal()->language_mode());
10.      if (info->is_eval()) {
11.        info->set_allow_eval_cache(parser.allow_eval_cache());
12.      }
13.    }
14.    if (mode == ReportErrorsAndStatisticsMode::kYes) {
15.  //omit.............................
16.    }
17.    return (result != nullptr);
18.  } In line 5, create the parser, namely, parser initialization. 1.  Parser::Parser(ParseInfo* info)
2.      : ParserBase<Parser>(info->zone(), &scanner_, info->stack_limit(),
3.                           info->extension(), info->GetOrCreateAstValueFactory(),
4.                           info->pending_error_handler(),
5.                           info->runtime_call_stats(), info->logger(),
6.                           info->script().is_null() ? -1 : info->script()->id(),
7.                           info->is_module(), true),
8.        info_(info),
9.        scanner_(info->character_stream(), info->is_module()),
10.        preparser_zone_(info->zone()->allocator(), ZONE_NAME),
11.        reusable_preparser_(nullptr),
12.        mode_(PARSE_EAGERLY),  // Lazy mode must be set explicitly.
13.        source_range_map_(info->source_range_map()),
14.        target_stack_(nullptr),
15.        total_preparse_skipped_(0),
16.        consumed_preparse_data_(info->consumed_preparse_data()),
17.        preparse_data_buffer_(),
18.        parameters_end_pos_(info->parameters_end_pos()) {
19.    bool can_compile_lazily = info->allow_lazy_compile() && !info->is_eager();
20.    set_default_eager_compile_hint(can_compile_lazily
21.                                       ? FunctionLiteral::kShouldLazyCompile
22.                                       : FunctionLiteral::kShouldEagerCompile);
23.    allow_lazy_ = info->allow_lazy_compile() && info->allow_lazy_parsing() &&
24.                  info->extension() == nullptr && can_compile_lazily;
25.    set_allow_natives(info->allow_natives_syntax());
26.    set_allow_harmony_dynamic_import(info->allow_harmony_dynamic_import());
27.    set_allow_harmony_import_meta(info->allow_harmony_import_meta());
28.    set_allow_harmony_nullish(info->allow_harmony_nullish());
29.    set_allow_harmony_optional_chaining(info->allow_harmony_optional_chaining());
30.    set_allow_harmony_private_methods(info->allow_harmony_private_methods());
31.    for (int feature = 0; feature < v8::Isolate::kUseCounterFeatureCount;
32.         ++feature) {
33.      use_counts_[feature] = 0;
34.    }
35.  } From line 8 to line 18, take out the compilation information, such as scanner, and compilation mode. From line 9 to line 23, the important stuff is can_compile_lazily. Line 25 enables the native commands that start with %. Summary V8 uses UTF16 to encode JavaScript source code; V8 uses v8::internal::source to manage our JavaScript code; First, look up the compilation cache, then start up the compiler if the cache miss; The scanner is passive and the parser is active. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : WeChat Email v8blink@outlook.com Also Published here

Understanding Chrome V8 - Chapter 20: How Compilers and Parsers Work

Welcome to other chapters of Let’s Understand Chrome V8 For performance reasons, V8 uses the compilation cache to hold the SharedFunction which is the compiler result, the cache returns the result directly once the same JavaScript code is compiled again. You may ask why the same code is compiled again. Compile first, then execute — this is the pipeline of V8. Even the exact same code, the same function, still needs to be sent to the compiler, but not always recompiled. Actually, JavaScript code may change during execution, so the first action always is that we get the JavaScript code’s hash-value from the compilation cache, return the SharedFunction if the cache is hit, and recompile the JavaScript conversely. 1. Initialization and Filling The Isolate, which is the V8 virtual machine instance, is responsible for initializing the CompilationCache, isolate::Init as shown below: bool Isolate::Init(ReadOnlyDeserializer* read_only_deserializer,
                   StartupDeserializer* startup_deserializer) {
//omit...................
#define ASSIGN_ELEMENT(CamelName, hacker_name)                  \
  isolate_addresses_[IsolateAddressId::k##CamelName##Address] = \
      reinterpret_cast<Address>(hacker_name##_address());
  FOR_EACH_ISOLATE_ADDRESS_NAME(ASSIGN_ELEMENT)
#undef ASSIGN_ELEMENT

  compilation_cache_ = new CompilationCache(this); The last line is now a CompilationCache, namely, initialization. Let’s dive into it in more depth. 1.  class V8_EXPORT_PRIVATE CompilationCache {
2.   public:
3.    MaybeHandle<SharedFunctionInfo> LookupScript(
4.        Handle<String> source, MaybeHandle<Object> name, int line_offset,
5.        int column_offset, ScriptOriginOptions resource_options,
6.        Handle<Context> native_context, LanguageMode language_mode);
7.    InfoCellPair LookupEval(Handle<String> source,
8.                            Handle<SharedFunctionInfo> outer_info,
9.                            Handle<Context> context, LanguageMode language_mode,
10.                            int position);
11.    MaybeHandle<FixedArray> LookupRegExp(Handle<String> source,
12.                                         JSRegExp::Flags flags);
13.    void PutScript(Handle<String> source, Handle<Context> native_context,
14.                   LanguageMode language_mode,
15.                   Handle<SharedFunctionInfo> function_info);
16.    void PutEval(Handle<String> source, Handle<SharedFunctionInfo> outer_info,
17.                 Handle<Context> context,
18.                 Handle<SharedFunctionInfo> function_info,
19.                 Handle<FeedbackCell> feedback_cell, int position);
20.    void PutRegExp(Handle<String> source, JSRegExp::Flags flags,
21.                   Handle<FixedArray> data);
22.    void Clear();
23.    void Remove(Handle<SharedFunctionInfo> function_info);
24.    void Iterate(RootVisitor* v);
25.    void MarkCompactPrologue();
26.    void Enable();
27.    void Disable();
28.   private:
29.    explicit CompilationCache(Isolate* isolate);
30.    ~CompilationCache() = default;
31.    base::HashMap* EagerOptimizingSet();
32.    static const int kSubCacheCount = 4;
33.    bool IsEnabled() const { return FLAG_compilation_cache && enabled_; }
34.    Isolate* isolate() const { return isolate_; }
35.    Isolate* isolate_;
36.    CompilationCacheScript script_;
37.    CompilationCacheEval eval_global_;
38.    CompilationCacheEval eval_contextual_;
39.    CompilationCacheRegExp reg_exp_;
40.    CompilationSubCache* subcaches_[kSubCacheCount];
41.    bool enabled_;
42.    friend class Isolate;
43.    DISALLOW_COPY_AND_ASSIGN(CompilationCache);
44.  }; There’re three fundamentals of the above function that are important: 1. LookupScript() at line 3. It is used to look up the cache to take out the SharedFunction that holds the bytecodes corresponding to your JavaScript code. If hit, skip compilation into the execution unit directly. 2. PutScript() at line 13. If LookupScript misses, PutScript() puts the hash corresponding to your JavaScript code and the SharedFunciton that the compiler generates into the cache before execution. 3. LookupEval and PutEval at lines 7 and 16. They are the same as LookupScript and PutScript but specialize in the JavaScript eval method because the eval needs not only SharedFunction but also context. Let’s examine LookupScript in a little more depth: 1.  MaybeHandle<SharedFunctionInfo> CompilationCache::LookupScript(
2.      Handle<String> source, MaybeHandle<Object> name, int line_offset,
3.      int column_offset, ScriptOriginOptions resource_options,
4.      Handle<Context> native_context, LanguageMode language_mode) {
5.    if (!IsEnabled()) return MaybeHandle<SharedFunctionInfo>();
6.    return script_.Lookup(source, name, line_offset, column_offset,
7.                          resource_options, native_context, language_mode);
8.  }
9.  //..................separation..........................
10.  //..................separation..........................
11.  MaybeHandle<SharedFunctionInfo> CompilationCacheScript::Lookup(
12.      Handle<String> source, MaybeHandle<Object> name, int line_offset,
13.      int column_offset, ScriptOriginOptions resource_options,
14.      Handle<Context> native_context, LanguageMode language_mode) {
15.    MaybeHandle<SharedFunctionInfo> result;
16.    {
17.      HandleScope scope(isolate());
18.      const int generation = 0;
19.      DCHECK_EQ(generations(), 1);
20.      Handle<CompilationCacheTable> table = GetTable(generation);
21.      MaybeHandle<SharedFunctionInfo> probe = CompilationCacheTable::LookupScript(
22.          table, source, native_context, language_mode);
23.      Handle<SharedFunctionInfo> function_info;
24.      if (probe.ToHandle(&function_info)) {
25.        if (HasOrigin(function_info, name, line_offset, column_offset,
26.                      resource_options)) {
27.          result = scope.CloseAndEscape(function_info);
28.        }
29.      }
30.    }
31.    Handle<SharedFunctionInfo> function_info;
32.    if (result.ToHandle(&function_info)) {
33.  #ifdef DEBUG
34.      DCHECK(HasOrigin(function_info, name, line_offset, column_offset,
35.                       resource_options));
36.  #endif
37.      isolate()->counters()->compilation_cache_hits()->Increment();
38.      LOG(isolate(), CompilationCacheEvent("hit", "script", *function_info));
39.    } else {
40.      isolate()->counters()->compilation_cache_misses()->Increment();
41.    }
42.    return result;
43.  } We can find that the CompilationCache::LookupScript is only the entrance, and the key function, as well as the important code, are all in CompilationCacheScript::Lookup(). 1. In Lookup(), the argument source is the JavaScript that you as developers are actually writing, which is going to be compiled. 2. GetTable at line 20. It is the actual table that holds data into which PutScript put. 3. CompilationCacheTable::LookupScript at line 21. Look up the table for a result in which hit means a SharedFunction or miss is null. 4. In line 25: Check if the SharedFunction is the correct one depending on Origin. Let’s take a glance at HasOrigin. 1.  // We only re-use a cached function for some script source code if the
2.  // script originates from the same place. This is to avoid issues
3.  // when reporting errors, etc.
4.  bool CompilationCacheScript::HasOrigin(Handle<SharedFunctionInfo> function_info,
5.                                         MaybeHandle<Object> maybe_name,
6.                                         int line_offset, int column_offset,
7.                                         ScriptOriginOptions resource_options) {
8.    Handle<Script> script =
9.        Handle<Script>(Script::cast(function_info->script()), isolate());
10.    Handle<Object> name;
11.    if (!maybe_name.ToHandle(&name)) {
12.      return script->name().IsUndefined(isolate());
13.    }
14.    if (line_offset != script->line_offset()) return false;
15.    if (column_offset != script->column_offset()) return false;
16.    if (!name->IsString() || !script->name().IsString()) return false;
17.    if (resource_options.Flags() != script->origin_options().Flags())
18.      return false;
19.    return String::Equals(
20.        isolate(), Handle<String>::cast(name),
21.        Handle<String>(String::cast(script->name()), isolate()));
22.  } From line 1 to line 3, the comments provide the origin check rules. Specifically, the implementation is from line 14 to line 16. Next, let’s go into PutScript. 1.  void CompilationCache::PutScript(Handle<String> source,
2.                                   Handle<Context> native_context,
3.                                   LanguageMode language_mode,
4.                                   Handle<SharedFunctionInfo> function_info) {
5.    if (!IsEnabled()) return;
6.    LOG(isolate(), CompilationCacheEvent("put", "script", *function_info));
7.    script_.Put(source, native_context, language_mode, function_info);
8.  }
9.  //.......................separation............................
10.  void CompilationCacheScript::Put(Handle<String> source,
11.                                   Handle<Context> native_context,
12.                                   LanguageMode language_mode,
13.                                   Handle<SharedFunctionInfo> function_info) {
14.    HandleScope scope(isolate());
15.    Handle<CompilationCacheTable> table = GetFirstTable();
16.    SetFirstTable(CompilationCacheTable::PutScript(table, source, native_context,
17.                                                   language_mode, function_info));
18.  } As same as LookupScript(), the CompilationCache::PutScript is entrance, all important stuff is in CompilationCacheTable::PutScript. In PutScript, I would like to inform you that the cache table will be filling. Below is the CacheTable source code: 1.  // This cache is used in two different variants. For regexp caching, it simply
2.  // maps identifying info of the regexp to the cached regexp object. Scripts and
3.  // eval code only gets cached after a second probe for the code object. To do
4.  // so, on first "put" only a hash identifying the source is entered into the
5.  // cache, mapping it to a lifetime count of the hash. On each call to Age all
6.  // such lifetimes get reduced, and removed once they reach zero. If a second put
7.  // is called while such a hash is live in the cache, the hash gets replaced by
8.  // an actual cache entry. Age also removes stale live entries from the cache.
9.  // Such entries are identified by SharedFunctionInfos pointing to either the
10.  // recompilation stub, or to "old" code. This avoids memory leaks due to
11.  // premature caching of scripts and eval strings that are never needed later.
12.  class CompilationCacheTable
13.      : public HashTable<CompilationCacheTable, CompilationCacheShape> {
14.   public:
15.    NEVER_READ_ONLY_SPACE
16.    static MaybeHandle<SharedFunctionInfo> LookupScript(
17.        Handle<CompilationCacheTable> table, Handle<String> src,
18.        Handle<Context> native_context, LanguageMode language_mode);
19.    static InfoCellPair LookupEval(Handle<CompilationCacheTable> table,
20.                                   Handle<String> src,
21.                                   Handle<SharedFunctionInfo> shared,
22.                                   Handle<Context> native_context,
23.                                   LanguageMode language_mode, int position);
24.    Handle<Object> LookupRegExp(Handle<String> source, JSRegExp::Flags flags);
25.    static Handle<CompilationCacheTable> PutScript(
26.        Handle<CompilationCacheTable> cache, Handle<String> src,
27.        Handle<Context> native_context, LanguageMode language_mode,
28.        Handle<SharedFunctionInfo> value);
29.//omit......................
30.    }; From line 1 to line 11, the comments provide more corresponding explanations enough. Let’s go back to CompilationCacheScript::Put() and ahead to line 16 which is CompilationCacheTable::PutScript(). 1.  Handle<CompilationCacheTable> CompilationCacheTable::PutScript(
2.      Handle<CompilationCacheTable> cache, Handle<String> src,
3.      Handle<Context> native_context, LanguageMode language_mode,
4.      Handle<SharedFunctionInfo> value) {
5.    Isolate* isolate = native_context->GetIsolate();
6.    Handle<SharedFunctionInfo> shared(native_context->empty_function().shared(),
7.                                      isolate);
8.    src = String::Flatten(isolate, src);
9.    StringSharedKey key(src, shared, language_mode, kNoSourcePosition);
10.    Handle<Object> k = key.AsHandle(isolate);
11.    cache = EnsureCapacity(isolate, cache, 1);
12.    int entry = cache->FindInsertionEntry(key.Hash());
13.    cache->set(EntryToIndex(entry), *k);
14.    cache->set(EntryToIndex(entry) + 1, *value);
15.    cache->ElementAdded();
16.    return cache;
17.  } I think the important thing is that PutScript binds the JavaScript source and SharedFuncation as a hash pair as well as puts it into the cache, which the implementations are from line 9 to line 15. 2. Lookup and update Let’s go into Compiler::GetSharedFunctionInfoForScript. 1.   MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
2.       Isolate* isolate, Handle<String> source,
3.       const Compiler::ScriptDetails& script_details,
4.       ScriptOriginOptions origin_options, v8::Extension* extension,
5.       ScriptData* cached_data, ScriptCompiler::CompileOptions compile_options,
6.       ScriptCompiler::NoCacheReason no_cache_reason, NativesFlag natives) {
7.  //omit.....................
8.       if (extension == nullptr) {
9.         bool can_consume_code_cache =
10.             compile_options == ScriptCompiler::kConsumeCodeCache;
11.         if (can_consume_code_cache) {
12.           compile_timer.set_consuming_code_cache();
13.         }
14.         maybe_result = compilation_cache->LookupScript(//lookup is here.
15.             source, script_details.name_obj, script_details.line_offset,
16.             script_details.column_offset, origin_options, isolate->native_context(),
17.             language_mode);
18.         if (!maybe_result.is_null()) {
19.           compile_timer.set_hit_isolate_cache();
20.         } else if (can_consume_code_cache) {
21.  //omit
22.           if (CodeSerializer::Deserialize(isolate, cached_data, source,
23.                                           origin_options)
24.                   .ToHandle(&inner_result) &&
25.               inner_result->is_compiled()) {
26.             is_compiled_scope = inner_result->is_compiled_scope();
27.             DCHECK(is_compiled_scope.is_compiled());
28.     		//filling is here
29.             compilation_cache->PutScript(source, isolate->native_context(),
30.                                          language_mode, inner_result);
31.             Handle<Script> script(Script::cast(inner_result->script()), isolate);
32.             maybe_result = inner_result;
33.           } else {
34.             compile_timer.set_consuming_code_cache_failed();
35.           }
36.         }
37.       }
38.       return maybe_result;
39.     } In line 14, it means that we need to compile the JavaScript code if LookupScript is missing, so we skip to line 21 to perform compilation and generate inner_result. Then at line 29, put the inner_result into the cache. We can find that the innuer_result is just the maybe_result at line 32. Figure 1 shows the call stack. Besides LookupScript and PutScript, there are also LookupEval, PutEval as well as LookupRegExp, PutRegExp in V8. The workflow of them is the same as LookupScript and PutScript, please learn by yourself. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. qq9123013 v8blink@outlook.com WeChat: Email: Also published . here

Understanding Chrome V8: Chapter 19: Compilation Cache — Make the Compiler Faster

Execution is responsible for executing the bytecode. In this paper, we’ll talk about the workflow from S to E and several kernel functions. Our test case: function JsPrint(a){
	if(a >5){
		return "Debug";
	}
}
console.log(JsPrint(6)); Workflow The GetSharedFunctionInfoForScript() compiles JS source code into bytecodes. 0.  MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
1.      Isolate* isolate, Handle<String> source,
2.      const Compiler::ScriptDetails& script_details,
3.      ScriptOriginOptions origin_options, v8::Extension* extension,
4.      ScriptData* cached_data, ScriptCompiler::CompileOptions compile_options,
5.      ScriptCompiler::NoCacheReason no_cache_reason, NativesFlag natives) {
6.  //.omit.............
7.    // Do a lookup in the compilation cache but not for extensions.
8.    MaybeHandle<SharedFunctionInfo> maybe_result;
9.    IsCompiledScope is_compiled_scope;
10.    if (extension == nullptr) {
11.      bool can_consume_code_cache =
12.          compile_options == ScriptCompiler::kConsumeCodeCache;
13.      if (can_consume_code_cache) {
14.        compile_timer.set_consuming_code_cache();
15.      }
16.      // First check per-isolate compilation cache.
17.      maybe_result = compilation_cache->LookupScript(
18.          source, script_details.name_obj, script_details.line_offset,
19.          script_details.column_offset, origin_options, isolate->native_context(),
20.          language_mode);
21.      if (!maybe_result.is_null()) {
22.        compile_timer.set_hit_isolate_cache();
23.      } 
24.    }
25.  //.....omit..............
26.    if (maybe_result.is_null()) {
27.      ParseInfo parse_info(isolate);
28.      // No cache entry found compile the script.
29.      NewScript(isolate, &parse_info, source, script_details, origin_options,
30.                natives);
31.      // Compile the function and add it to the isolate cache.
32.      if (origin_options.IsModule()) parse_info.set_module();
33.      parse_info.set_extension(extension);
34.      parse_info.set_eager(compile_options == ScriptCompiler::kEagerCompile);
35.      parse_info.set_language_mode(
36.          stricter_language_mode(parse_info.language_mode(), language_mode));
37.      maybe_result = CompileToplevel(&parse_info, isolate, &is_compiled_scope);
38.      Handle<SharedFunctionInfo> result;
39.      if (extension == nullptr && maybe_result.ToHandle(&result)) {
40.        DCHECK(is_compiled_scope.is_compiled());
41.        compilation_cache->PutScript(source, isolate->native_context(),
42.                                     language_mode, result);
43.      } else if (maybe_result.is_null() && natives != EXTENSION_CODE) {
44.        isolate->ReportPendingMessages();
45.      }
46.    }
47.    return maybe_result;
48.  } In the 17th line of code, lookup compilation_cache, cache hit indicates that there are already bytecodes, and don’t need to recompile. The 26th line of code indicates that cache miss, so call the CompileToplevel on the 37th line to generate bytecodes. Bytecode needs to be bound to a context before execution, as below: 0.  Local<Script> UnboundScript::BindToCurrentContext() {
1.    auto function_info =
2.        i::Handle<i::SharedFunctionInfo>::cast(Utils::OpenHandle(this));
3.    i::Isolate* isolate = function_info->GetIsolate();
4.    i::Handle<i::JSFunction> function =
5.        isolate->factory()->NewFunctionFromSharedFunctionInfo(
6.            function_info, isolate->native_context());
7.    return ToApiHandle<Script>(function);
8.  } After binding, we get an important member — JSFunction, which is the input expected by the execution. 0.  bool Shell::ExecuteString(Isolate* isolate, Local<String> source,
1.                            Local<Value> name, PrintResult print_result,
2.                            ReportExceptions report_exceptions,
3.                            ProcessMessageQueue process_message_queue) {
4.    bool success = true;
5.    {
6.      PerIsolateData* data = PerIsolateData::Get(isolate);
7.      Local<Context> realm =
8.          Local<Context>::New(isolate, data->realms_[data->realm_current_]);
9.      Context::Scope context_scope(realm);
10.      MaybeLocal<Script> maybe_script;
11.      Local<Context> context(isolate->GetCurrentContext());
12.      ScriptOrigin origin(name);
13.      if (options.compile_options == ScriptCompiler::kConsumeCodeCache) {
14.  //.......omit
15.      } else if (options.stress_background_compile) {
16.  //.......omit
17.      } else {
18.        ScriptCompiler::Source script_source(source, origin);
19.        maybe_script = ScriptCompiler::Compile(context, &script_source,
20.                                               options.compile_options);
21.      }
22.      maybe_result = script->Run(realm);
23.  //......omit
24.      if (options.code_cache_options ==
25.          ShellOptions::CodeCacheOptions::kProduceCacheAfterExecute) {
26.        // Serialize and store it in memory for the next execution.
27.        ScriptCompiler::CachedData* cached_data =
28.            ScriptCompiler::CreateCodeCache(script->GetUnboundScript());
29.        StoreInCodeCache(isolate, source, cached_data);
30.        delete cached_data;
31.      }
32.      if (process_message_queue && !EmptyMessageQueues(isolate)) success = false;
33.      data->realm_current_ = data->realm_switch_;
34.    }
35.    DCHECK(!try_catch.HasCaught());
36.    return success;
37.  } The above function is the entrance for executing JS. Line 19 compiles the JS to bytecode. Line 22 executes the bytecode. Debug line 22 to step into the Run(realm). 0.  MaybeLocal<Value> Script::Run(Local<Context> context) {
1.    auto isolate = reinterpret_cast<i::Isolate*>(context->GetIsolate());
2.    TRACE_EVENT_CALL_STATS_SCOPED(isolate, "v8", "V8.Execute");
3.    ENTER_V8(isolate, context, Script, Run, MaybeLocal<Value>(),
4.             InternalEscapableScope);
5.    i::HistogramTimerScope execute_timer(isolate->counters()->execute(), true);
6.    i::AggregatingHistogramTimerScope timer(isolate->counters()->compile_lazy());
7.    i::TimerEventScope<i::TimerEventExecute> timer_scope(isolate);
8.    auto fun = i::Handle<i::JSFunction>::cast(Utils::OpenHandle(this));
9.    i::Handle<i::Object> receiver = isolate->global_proxy();
10.    Local<Value> result;
11.    has_pending_exception = !ToLocal<Value>(
12.        i::Execution::Call(isolate, fun, receiver, 0, nullptr), &result);
13.    RETURN_ON_FAILED_EXECUTION(Value);
14.    RETURN_ESCAPED(result);
15.  } The 8th line casts this pointer to JSFunction. For our case, the variable fun is console.log(JsPrint(6)), then call the 12th line — call(). In call(), the Invoke() will be called which is below. 0.  V8_WARN_UNUSED_RESULT MaybeHandle<Object> Invoke(Isolate* isolate,
1.                                                   const InvokeParams& params) {
2.  //...............omit
3.    Object value;
4.    Handle<Code> code =
5.        JSEntry(isolate, params.execution_target, params.is_construct);
6.    {
7.      SaveContext save(isolate);
8.      SealHandleScope shs(isolate);
9.      if (FLAG_clear_exceptions_on_js_entry) isolate->clear_pending_exception();
10.      if (params.execution_target == Execution::Target::kCallable) {
11.        // clang-format off
12.        // {new_target}, {target}, {receiver}, return value: tagged pointers
13.        // {argv}: pointer to array of tagged pointers
14.        using JSEntryFunction = GeneratedCode<Address(
15.            Address root_register_value, Address new_target, Address target,
16.            Address receiver, intptr_t argc, Address** argv)>;
17.        // clang-format on
18.        JSEntryFunction stub_entry =
19.            JSEntryFunction::FromAddress(isolate, code->InstructionStart());
20.        Address orig_func = params.new_target->ptr();
21.        Address func = params.target->ptr();
22.        Address recv = params.receiver->ptr();
23.        Address** argv = reinterpret_cast<Address**>(params.argv);
24.        RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::kJS_Execution);
25.        value = Object(stub_entry.Call(isolate->isolate_data()->isolate_root(),
26.                                       orig_func, func, recv, params.argc, argv));
27.      } else {
28.  //..................omit     
29.      }
30.    } Line 18 gets the address of the builtin, its index is 40, and its name is JSEntry. This builtin builds the execution environment for bytecodes. Then step into line 25, where the bytecode is executed. After that, we can only debug the assembly code, but c++. Figure 1 is the call stack. Before stepping into bytecodes, Generate_JSEntryVariant will be called. 0.  void Generate_JSEntryVariant(MacroAssembler* masm, StackFrame::Type type,
1.                               Builtins::Name entry_trampoline) {
2.    Label invoke, handler_entry, exit;
3.    Label not_outermost_js, not_outermost_js_2;
4.    {  // NOLINT. Scope block confuses linter.
5.      NoRootArrayScope uninitialized_root_register(masm);
6.      // Set up frame.
7.      __ pushq(rbp);
8.      __ movq(rbp, rsp);
9.      // Push the stack frame type.
10.      __ Push(Immediate(StackFrame::TypeToMarker(type)));
11.      // Reserve a slot for the context. It is filled after the root register has
12.      // been set up.
13.      __ AllocateStackSpace(kSystemPointerSize);
14.      // Save callee-saved registers (X64/X32/Win64 calling conventions).
15.      __ pushq(r12);
16.      __ pushq(r13);
17.      __ pushq(r14);
18.      __ pushq(r15);
19.  #ifdef _WIN64
20.      __ pushq(rdi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
21.      __ pushq(rsi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
22.  #endif
23.      __ pushq(rbx);
24.  //.....................omit............................
25.    __ ret(0);
26.  }
27.  }  // namespace
28.  //==================separation===============================
29.  void Builtins::Generate_JSEntry(MacroAssembler* masm) {
30.    Generate_JSEntryVariant(masm, StackFrame::ENTRY,
31.                            Builtins::kJSEntryTrampoline);
32.  } Look at the 30th line of code, the generator function of the 40th builtin is B, and its analysis method refers to my previous paper. Figure 2 shows the corresponding assembly code. You can also see a bit of C++ in Figure 2, but this is the last chance to see C++. After that, only the assembly code. Our case uses the console.log which is a global function. It is below. 1.  #define RUNTIME_FUNCTION_RETURNS_TYPE(Type, InternalType, Convert, Name)      \
2.    static V8_INLINE InternalType __RT_impl_##Name(Arguments args,              \
3.                                                   Isolate* isolate);           \
4.                                                                                \
5.    V8_NOINLINE static Type Stats_##Name(int args_length, Address* args_object, \
6.                                         Isolate* isolate) {                    \
7.      RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::k##Name);      \
8.      TRACE_EVENT0(TRACE_DISABLED_BY_DEFAULT("v8.runtime"),                     \
9.                   "V8.Runtime_" #Name);                                        \
10.     Arguments args(args_length, args_object);                                 \
11.      return Convert(__RT_impl_##Name(args, isolate));                          \
12.    }                                                                           \
13.                                                                                \
14.    Type Name(int args_length, Address* args_object, Isolate* isolate) {        \
15.      DCHECK(isolate->context().is_null() || isolate->context().IsContext());   \
16.      CLOBBER_DOUBLE_REGISTERS();                                               \
17.      if (V8_UNLIKELY(TracingFlags::is_runtime_stats_enabled())) {              \
18.        return Stats_##Name(args_length, args_object, isolate);                 \
19.      }                                                                         \
20.      Arguments args(args_length, args_object);                                 \
21.      return Convert(__RT_impl_##Name(args, isolate));                          \
22.    }                                                                           \
23.                                                                                \
24.    static InternalType __RT_impl_##Name(Arguments args, Isolate* isolate)
25.  #define CONVERT_OBJECT(x) (x).ptr()
26.  #define CONVERT_OBJECTPAIR(x) (x)
27.  #define RUNTIME_FUNCTION(Name) \
28.    RUNTIME_FUNCTION_RETURNS_TYPE(Address, Object, CONVERT_OBJECT, Name)
29.  //=================separation==============================
30.  //==================separation===============================
31.  Object DeclareGlobals(Isolate* isolate, Handle<FixedArray> declarations,
32.                        int flags, Handle<JSFunction> closure) {
33.    HandleScope scope(isolate);
34.    Handle<JSGlobalObject> global(isolate->global_object());
35.    Handle<Context> context(isolate->context(), isolate);
36.    Handle<FeedbackVector> feedback_vector = Handle<FeedbackVector>::null();
37.    Handle<ClosureFeedbackCellArray> closure_feedback_cell_array =
38.        Handle<ClosureFeedbackCellArray>::null();
39.    if (closure->has_feedback_vector()) {
40.      feedback_vector =
41.          Handle<FeedbackVector>(closure->feedback_vector(), isolate);
42.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
43.          feedback_vector->closure_feedback_cell_array(), isolate);
44.    } else {
45.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
46.          closure->closure_feedback_cell_array(), isolate);
47.    }
48.    // Traverse the name/value pairs and set the properties.
49.    int length = declarations->length();
50.    FOR_WITH_HANDLE_SCOPE(isolate, int, i = 0, i, i < length, i += 4, {
51.      Handle<String> name(String::cast(declarations->get(i)), isolate);
52.      FeedbackSlot slot(Smi::ToInt(declarations->get(i + 1)));
53.      Handle<Object> possibly_feedback_cell_slot(declarations->get(i + 2),
54.                                                 isolate);
55.      Handle<Object> initial_value(declarations->get(i + 3), isolate);
56.      bool is_var = initial_value->IsUndefined(isolate);
57.      bool is_function = initial_value->IsSharedFunctionInfo();
58.      DCHECK_NE(is_var, is_function);
59.      Handle<Object> value;
60.      if (is_function) {
61.        DCHECK(possibly_feedback_cell_slot->IsSmi());
62.        Handle<FeedbackCell> feedback_cell =
63.            closure_feedback_cell_array->GetFeedbackCell(
64.                Smi::ToInt(*possibly_feedback_cell_slot));
65.        // Copy the function and update its context. Use it as value.
66.        Handle<SharedFunctionInfo> shared =
67.            Handle<SharedFunctionInfo>::cast(initial_value);
68.        Handle<JSFunction> function =
69.            isolate->factory()->NewFunctionFromSharedFunctionInfo(
70.                shared, context, feedback_cell, AllocationType::kOld);
71.        value = function;
72.      } else {
73.        value = isolate->factory()->undefined_value();
74.      }
75.      // Compute the property attributes. According to ECMA-262,
76.      // the property must be non-configurable except in eval.
77.      bool is_eval = DeclareGlobalsEvalFlag::decode(flags);
78.      int attr = NONE;
79.      if (!is_eval) attr |= DONT_DELETE;
80.      // ES#sec-globaldeclarationinstantiation 5.d:
81.      // If hasRestrictedGlobal is true, throw a SyntaxError exception.
82.      Object result = DeclareGlobal(isolate, global, name, value,
83.                                    static_cast<PropertyAttributes>(attr), is_var,
84.                                    is_function, RedeclarationType::kSyntaxError,
85.                                    feedback_vector, slot);
86.      if (isolate->has_pending_exception()) return result;
87.    });
88.    return ReadOnlyRoots(isolate).undefined_value();
89.  }
90.  //==================separation===============================
91.  //==================separation===============================
92.  RUNTIME_FUNCTION(Runtime_DeclareGlobals) {
93.    HandleScope scope(isolate);
94.    DCHECK_EQ(3, args.length());
95.    CONVERT_ARG_HANDLE_CHECKED(FixedArray, declarations, 0);
96.    CONVERT_SMI_ARG_CHECKED(flags, 1);
97.    CONVERT_ARG_HANDLE_CHECKED(JSFunction, closure, 2);
98.    return DeclareGlobals(isolate, declarations, flags, closure);
99.  } RUNTIME_FUNCTION(Runtime_DeclareGlobals) is a runtime function defined by a macro template. The function checks the parameters and then calls DeclareGlobals() to perform the corresponding operation. In our case, DeclareGlobals() lookup the global object console, then takes out the log() from the console, and gets the address of console.log(). Finally, pass the parameter JsPrint(6) into log() which will start a new compilation and execution. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : v8blink@outlook.com WeChat Email This story was first published here.

Let’s Understand Chrome V8 — Chapter 10: Ignition Execution Unit

The State of the Russia-Ukraine War from a Ukrainian HackerNooner

How to Implement Micro Frontend Architecture Based on Angular

Cómo implementar una arquitectura Micro Frontend basada en Angular

10 Essential Javascript Questions

Photo by on Sergey Pesterev Unsplash is the prototypical response when asking a software engineer for advice, no matter how straight-forward the problem may seem. When asked “Should we choose TypeScript or Flow for our next React Native project?” however, the answer only depends on one variable: . “Well, it depends” whether or not you work at Facebook It’s interesting to consider how we got here. The project team evaluated Flow vs TypeScript for our new React Native app almost three years ago. At the time, TypeScript didn’t support React well, didn’t allow for a gradual opt-in, there was no Babel support, and VSCode wasn’t the editor providing the best JavaScript development experience on the market. None of these factors are true today. But why bother to migrate? TypeScript has a lot of value to offer making the switch highly attractive: More “future proof” Higher OSS adoption & better RN + library support Awesome developer experience (VSCode) Consistency with other department projects Easier to hire for So, we forged ahead! But how’d it go? Before: 50K SLOC, 84% Flow Coverage $ npx sloc app
---------- Result ------------
Physical :  57903
Source :  50859
Number of files :  709
----------------------------
$ npx flow-coverage-report  -i percent: 84 %
$ time flow check
flow 3.64s user 2.31s system 32% cpu 18.458 total read 'app/**/*.js' Getting Started The first step we took was to update our JS tooling configuration. For our project, the procedure was relatively straight-forward, guided mostly by simply copying the configurations directly from a newly instantiated TypeScript template project using the . react-native cli Prettier & ESLint { : } "parser" "typescript" One used to have to deal with TSLint when chosing TypeScript but thankfully , so it’s an easy choice in 2019—just stick with ESLint! TSLint is deprecated Literally no changes necessary (for our usage anyway, and that included plugins and custom rules!) Done. Next. Bonus Aside: Removing Black Magic module imports Did you know that you can import JavaScript modules relative to a package.json file using the name attribute in the json file, no plugins required? Black magic! But it was useful black magic! To accomplish absolute path imports (i.e. we included in our project a in the (and several others as well for a total of 4 package.json’s): import Foo from ‘app/ui/components/Foo') package.json /app directory { : } "name" "app" This even works when importing from other folders in the tree (meaning you can import from outside of the ‘app’ folder). app/components/Foo Well, TypeScript doesn’t like this black magic fuckery and couldn’t figure out our imports :(. So, we deleted our extra package.json files and chose to use babel instead. Babel With built in Babel support for TypeScript in Babel 7, converting our babel configuration from Flow to TypeScript was a process of simply removing unused plugins (like ). @babel/plugin-transform-flow-strip-types Here’s our final Babel.config.js, complete with less magical module resolution. .exports = { : [ ], : [
        [ ,
            { : [ ], : { : },
            },
        ],
    ]
} module presets 'module:metro-react-native-babel-preset' plugins 'module-resolver' root '.' alias app './app' tsconfig.json While not strictly required for running your React Native app (Babel will just strip/ignore the TS syntax anyway), the tsconfig is required for using the TypeScript compiler (and VSCode tooling) to detect type errors. Thankfully, we got 90% of the way there by simply copy-pasting the tsconfig from a boilerplate new React Native project. However, we also include some files in our project, which require some additional configuration. .json { : { : , : , : , : , : , : , : [ , ], : , : , : , : , : , : }, : [ , , , ,
  ]
} "compilerOptions" "baseUrl" "." "allowJs" true "allowSyntheticDefaultImports" true "esModuleInterop" true "isolatedModules" true "jsx" "react-native" "lib" "es6" "dom" "moduleResolution" "node" "noEmit" true "strict" true "target" "esnext" "resolveJsonModule" true // Required for JSON files "skipLibCheck" true "exclude" "node_modules" "e2e" "**/*.json" // Don't try and check JSON files "**/*.spec.ts" Migrating Flow to TS syntax and .js to .ts/x Thankfully some work has been done in the open source community to aid in the conversion from Flow to TypeScript syntax, and the projects have support for many of the common language features. However, I cannot say that these projects tend to be very well maintained or mainstream. There are two dominant solutions available, and . We tried the flow-to-typescript library first but abandoned it because it crashed on any file containing a function ( ). flow-to-typescript babel-plugin-flow-to-typescript WTF? Thankfully the babel-plugin-flow-to-typescript worked for us. Almost. As of writing the same related to function support forced us to use . I don’t know what’s up with these tools and supporting plain old functions but whatever, it worked, I guess I should be thankful! bug a fork Convert JavaScript + Flow with a .js extension to TypeScript with .ts: yarn add global @babel/cli @babel/core babel-plugin-flow-to-typescript npx babel script.js -o script.ts --plugins=babel-plugin-flow-to-typescript find app - f -name | parallel # Convert single file # Convert all files and delete original JS after conversion # Prereq: 'brew install parallel' type '*.js' "npx babel {} -o {.}.ts && rm {}" Note: We actually used @zxbodya/babel-plugin-flow-to-typescript due to a bug in the main repo. Next, rename any files which import React at the top to use .tsx instead: find app/components -name - sh -c {} \; "*.ts" exec 'git mv "$0" "${0%.ts}.tsx"' Sanity Check: Does the app still run? Before proceeding any further, it’s worth checking at this point that the project actually runs. Because Babel is used to compile TypeScript to JavaScript, and Babel simply strips out all TypeScript related syntax, regardless of how many type errors your project has it should “Just Work.” In our case, it took less than half a day to update configurations, rename files, migrate all syntax, and get a running app again. But the work is obviously still far from over. Fixing 1000s of compilation errors In our case, the effort began with ~1800 TS compilation errors. Fixing this amount of errors is not a linear journey where one fix means your errors go down to 1799. Sometimes, making a change will result in dozens fewer errors, making you feel like a god among mortals, and sometimes it increases your overall error count leaving your confidence shaken! #1—Fixing the Implicit Any The number one thing TypeScript will complain about is an implicit any. Mostly due to differences between TS and Flow regarding type inference, one must immediately add additional annotations. Be warned: . Instead, ask don’t go simply adding a type explicitly where TypeScript complains “Why is this thing any?” reqs = get(props, , []) inProgressIds = reqs.map( req.id) activeRequests = actions .filter( inProgressIds.includes(action.id)) // Problem here const 'requests' const => req const // Error shown here => action Rather than adding a type to as the error suggests, is missing a type, producing the error below. The root cause could be another issue in the function, or even an entirely different file! action reqs #2—Updating React components Easily the least enjoyable and most “hairy” part of the conversion is migrating React code. Errors thrown in React code often have long error chains, sometimes with details that seem at the surface totally unrelated to the required fix. Our project limits the use of React Native primitives from our “smart” components, choosing instead to create our own set of primitives in a reusable UI framework. Button RNButton type PropTypes = {|
    onPress: mixed, : boolean,
|} Button = <RNButton {...props } /> /* @flow */ import as from 'react-native' // Seal to avoid warning about inexact spread => () custom // Trivialized component const ( ) => props: PropTypes Under Flow, one must add every prop passed into to , where instead we’d prefer to take all the props that does and add in our own custom one. <Button> PropTypes <RNButton> Button RNButton, { ButtonProps } interface PropTypes extends ButtonProps { : boolean,
} Button = <RNButton {...props } /> // Import props available for every React Native component!  :D import as from 'react-native' custom const ( ) => props: PropTypes Much preferable (and accurate)! Interfaces are one of the best parts about TypeScript! #3—Refactoring HOCs One benefit gained with TypeScript over Flow is the ability to use generic type arguments inside the body of your function. Not sure how advisable generally this is, but it’s a great feature to have when adding types for higher order components! withAThing = <T>) => 
    (WrappedComponent: React.ComponentType<T>) => 
        (props: T) => <WrappedComponent {...props } /> // TS allows the use of generics in the body of your HOC.  Yes! const (config: Config < > T However, in order to leverage this feature, we more-or-less needed to completely rewrite our type definitions for all of our higher order components. Ouch. Needless to say we‘d prefer hooks! After: 96% TypeScript Coverage $ -coverage -p tsconfig.json 68712 / 71471 96.13%
$ time tsc
tsc  29.82s user 1.76s system 166% cpu 18.955 total type # ignored tests + storybook Coverage increase of ~12%. Execution time more or less identical. By far the most significant type coverage gaps exist in our Redux Saga code. This was true under Flow as well, but it seems switching to TypeScript hasn’t improved matters much, if any. Summary If you’re still using Flow with React Native, it’s never been easier to switch! It took the equivalent of 3 engineers working 10 full working days to accomplish the conversion, including time taken for regression testing, code review, refactoring, and developing new usage patterns. The majority of errors can be easily solved with a google search for the error message, while a few required time, consideration, and experience to be confident in the fix. Though we discovered a few bugs (mostly related to incorrect third party API usage), this was a minor benefit to the effort. Gaining some additional coverage on our React code did allow for some dead code elimination. Overall, we achieved a codebase better poised for ongoing future development both in community support and developer experience.

Migrating a 50K SLOC Flow + React Native app to TypeScript

How is a new era of data challenges changing operation and maintenance for IT? Best practice form the Alibaba tech team. This article is part of the AIOps for Big Data series. The heart of artificial intelligence is data, and companies with a lot of it are constantly working to boost what it can do for them. Beyond that simple fact, though, many continue to wonder what it means to say that data is involved in development practices like AIOps (AI for IT operations), or how data can be used in place of human and even machine-driven analytics. Further questions abound. How can we use machine learning algorithms together with big data-based business operation and maintenance platforms? How can machine learning enhance alarm filtering, anomaly monitoring, automated repairs, and other tasks to truly liberate operation and maintenance? Faced with these questions, Alibaba is moving away from a belief in AIOps as a long-term evolution toward a data-centric approach to IT. In that spirit, the group has departed from common industry practices and invested in a robust foundation in DataOps — an automated, process-oriented methodology that data analysts use to improve the quality and rate of analysis cycles. In this article, we look at the challenges and opportunities facing operation and maintenance teams as they move beyond outdated practices and into the data-driven era. From ScriptOps to AIOps, Level by Level L1: ScriptOps Scripted operation and maintenance Script replaces manual operation Execution: human + script Decision-making: human L2: ToolOps Automated operation and maintenance Most operation and maintenance work is done automatically or by processes Execution: human + system Decision-making: human L3: DevOps Highly automated + single-point intelligence Operation and maintenance is done by data system construction Execution: human + system (80%) Decision-making: human + system (20%) L4: DataOps (advanced) Highly automated + series intelligence Main operation and maintenance scenes are implemented by processes and free of intervention Execution: human + system (95%) Decision-making: human + system (80%) L5: AIOps Fully automatic smart operation and maintenance Can be easily adjusted between cost, quality, and efficiency Execution: system (100%) Decision-making: human + system (95%) Rough Beginnings: ScriptOps Operation and maintenance work requires a high level of skill, and the scope of the work exceeds other IT fields. Nevertheless, many think of it as limited to releases, modifications, alerts, and device migration, generally reflecting dated practices known together as ScriptOps. In some ways, this is not a bad sense to have. All big Internet companies begin as small companies where these issues (and all manner of other problems) threaten the company’s survival. Pressure and the pursuit of short-term results, though, have led many to rely on simplistic solutions from online technical forums or even personal blogs, leaving a legacy of misunderstanding that today’s professionals must move beyond. A Case for ToolOps The view described above is more than an outside misconception. Anyone who has led newcomers in the field is likely aware of their tendency to deploy one-click batch release software, one-click cleanup, interactive wizard execution, or other “black screen” scripts. Often, they simply re-implement some such solution according to their personal sense of it, failing to grasp the potential for mishap in different scenarios. This invites inefficiencies and security risks, and the history of the Internet is riddled with the disastrous consequences of mistakes as simple as typing in the wrong characters. Today, it is better understood that novices should not be left to run free on systems they have a limited grasp of. Instead, there is an ongoing push to merge more and more functional scripts into workable tools that can ensure the effective handover of the capabilities they provide — ToolOps, for short. Shifting to Platform-Based DevOps When an Internet company’s commercial success raises the scale of its operations, quantitative changes begin to create qualitative changes at the data level. Today, operation and maintenance for a large factory setting demands entirely new computing practices, and simply adding staff is not a solution. Put another way, when an application grows from hundreds of platform units to tens or hundreds of thousands, data processing changes from a simple matter of CPU, memory, and mechanical hard disks to an elaborate mix of GPUs, FPGAs, ASICs, Optane SSDs, and other hardware, software, and big data distributions. As issues threaten a large platform’s business and resources, data workers often face tasks bordering on the impossible. At such times, the operation and maintenance job description more closely resembles: · Global architecture planning · Resource operation and cost optimization · Automated platform development · Stability protection · Massive data analysis · Any number of unforeseen scenarios… For Alibaba, developing platforms to assist operation and maintenance workers in these cases is now a given. Entering the DataOps Phase As Alibaba’s business grows, its operation and maintenance capabilities have likewise grown in depth and precision. Through software engineering and data-based innovation, operation and maintenance tools must adapt to handle ultra-large-scale distributed cluster management and improve the stability, efficiency, and cost of the overall product. This presents tremendous challenges for operation and maintenance personnel and sets a high requirement for skill on their part. Simultaneously, the broader industry has also evolved toward a prevailing concept of AIOps. The field as a whole is pushing for greater awareness of these practices, driven by the idea that a powerful algorithm can replace the intelligence now afforded by human labor. Total automation is still an ambitious goal beyond today’s reality, much like driverless transit. At Alibaba, the prevailing thought is that if an algorithm is the kernel, then its value depends on the amount of engineering devoted to implementing it. This essentially describes the thinking behind the DataOps stage, in which data figures in all operation and maintenance goals, and data-driven operation and maintenance has been effectively implemented. The following image illustrates the aforementioned comparison to autonomous driving. Transforming Teams In the big data era, operation and maintenance personnel need to cultivate a new set of skills for a transforming industry. Key abilities now include architectural skills, research and development, operation and maintenance business sense, algorithm engineering, and TPM (technical project management) capability. Developing an effective AIOps is essential to Alibaba’s operation and maintenance platforms and products. Beyond initial construction, human input and participation will continue to define progress toward a working model for the long term. This requires a large number of workers, as well as experts and analysts of different backgrounds and business calibers. Beyond people, it involves supporting visualization technology, machine learning technology, big data analysis, development scenario analysis, and platform landing, all contributing to the ultimate business value developers are seeking. New trend of operation and maintenance Transforming an entire operation and maintenance team is often a painful process. Changing qualifications naturally bring about organizational changes, and the impact on veteran personnel is relatively high. From maintenance to research and development, the first changes are always ideological. In technical innovation, there are always initial principles, followed by a gradual implementation of the new project, while traditional operation and maintenance workers ensure ongoing stability. From putting out fires to transforming operations, this kind of pain is also part of the transformation any team has to go through. Ultimately, this should be a shift from problem-driven work to value-driven work, from operation and maintenance labor to operation and maintenance development, and from relying on experience to relying on smarter insight. It means the transformation not only of technological capabilities but also of operational and systemic thinking, in the spirit of embracing change and evolving with the times. (Original article by Ke Min柯旻) This article is part of the Big Data AIOps series. Alibaba Tech First hand and in-depth information about Alibaba’s latest technology → Facebook: . Twitter: . “Alibaba Tech” “AlibabaTech”

Even Smarter: Achieving AIOps in the Age of Big Data

The online marketing industry is broken, its reputation is bad and getting worse. Our industry is full of marketers, entrepreneurs and developers with , relying on extracting user data to fuel their unethical models. questionable styles, tactics and ethics driven by numbers business Growth hacks at cost of everything else Built with the purpose of taking attention away and making the users addicted to the products Built with the purpose being to collect data, sell that data and sell advertising Featuring dark patterns with design choices that to take actions that benefit the platform rather than themselves manipulate the users Tens if not hundreds of external, third-party scripts tracking every move Intentional in order to make it more difficult for users to quit, cancel, edit settings friction within design Buying, selling and scraping of user data such as email addresses with the intention of sending unsolicited mass sales mailers The most celebrated companies are the worst offenders Some of the most celebrated companies and people working within them are the worst offenders. Google has turned away from its “ ” motto and philosophy. don’t be evil You’ve already heard everything you need to know about how Facebook abuses your trust. The list is too long to include here. From the book Chaos Monkeys by , a former product manager at the company, you have a good insight into how a typical marketer thinks these days: Antonio García Martínez For my entire career at Facebook, I was embroiled in a rolling debate with the Facebook privacy and legal teams about what we could and couldn’t get away with, chiseling away at their legal trepidation, and trying to find some legal rubric that would forgive (or at least defensibly excuse) our next depredation with user data. Here’s a very current example of a dark pattern Facebook introduced in their design. The focus is on the fake new notification and message buttons on the top even though there actually are no new messages. The idea is to get more people to rush in and accept their invasive terms without a second thought. GDPR is just another obstacle for their growth team and something that can be hacked in order to manipulate more users to say yes to something they may not want. Despite what Mark Zuckerberg wants you or the senators to believe. Unilever is one of the largest advertisers online and this is : how they think Prowling his London office in jeans, Keith Weed, who oversees marketing and Unilever, one of the world’s largest advertisers, described how mobile phones have elevated data as a marketing tool. “Now with the good old mobile, I have individualized data on people. You don’t need to know their names . . . You know their telephone number. You know where they live, because it’s the same location as their PC.” Weed knows what times of the day you usually browse, watch videos, answer e-mail, travel to the office — and what travel routes you take. “From your mobile, I know whether you stay in four-star or two-star hotels, whether you go to train stations or airports. I use these insights along with what you’re browsing on your PC. I know whether you’re interested in horses or holidays in the Caribbean. The core values most big businesses (and those who are trying to become big) hold and stand for are the financial growth and monetary success. This growth matters the most and we use whatever tactic or strategy that leads us to it. This has attracted financially motivated people. And it’s difficult for these people to sacrifice short-term growth or financial success in exchange for user privacy, respect and strong ethics. ” It is difficult to get a man to understand something, when his salary depends upon his not understanding it.” This is the environment we’re working in. Discipline and patience are not tolerated. All the good values have costs, are viewed as barriers and are seen as a , at least in the short-term. competitive disadvantage GDPR has made advertising more complicated But the world we live and operate in is slowly changing. Social media and website advertising has gotten a lot more complicated since the General Data Protection Regulation (GDPR) has gone into effect. You now need permission from users in order to collect and use their personal data. All the GDPR notices on the different websites have really opened our eyes to how many third-party trackers are collecting our data. Compare and . The first is a typical website that you are used to these days. https://www.usatoday.com/ https://eu.usatoday.com/ It includes banners, intrusive advertising, it’s slow to load and uses a lot of your data. Behind the scenes it loads many external tracking scripts that are there to monitor your every move (on site and off site too), learn more about you so they can interrupt you to sell you more stuff you don’t need. More than 5MB of junk in total is loaded. The second version is the EU version which is compatible with the GDPR law. It’s pleasing to the eye, it’s quick to load, it doesn’t use much of your data, it doesn’t push intrusive ads you don’t want to see and it doesn’t load any tracking scripts in the background. The total size of the GDPR-friendly version is 500KB. It’s refreshing to visit, has a great user experience and is something that pretty much the whole web used to be before the big digital advertising revolution. With new ideas and better business models, I hope we can return to this type of ethical websites that are good for users but also companies running them. A growing number of marketers and companies with integrity Luckily there are a growing numbers of marketers and companies (and politicians with that support users) who share a wish to make the world of online a better place. even more regulations marketing . People with integrity. Something more people and companies should aspire to themselves. They hold admirable traits, qualities and a fresh, less intrusive view on how marketing should be done of recently voiced his opinion on this: Amir Salihefendic Doist of Farnam Street has recently updated his ethics and privacy policy plus removed all the Facebook tracking pixels: Shane Parrish , director of marketing at , : Kevan Lee Buffer is in on it too Good marketing requires empathy. Be genuine. Help people. Provide value first. Don’t persuade. Don’t sell someone something they don’t need. Enrich people’s lives. When they need you, be ready. Listen. Communicate clearly. Give freely. Advertise minimally. Seek permission and celebrate intent. Always speak the truth. Dream big. Deliver high quality. When in doubt, be a magnet not a hammer. Here’s an example of how things can be done from the book by , founder of Moz: Lost and Founder Rand Fishkin “You think we should make it harder to cancel?” I replied. “If you change the cancellation process to a phone call,” Rob explained, “I’m willing to bet that friction alone will improve your monthly churn rate. Plus, you can then talk to folks as they’re canceling and get a much better understanding of who they are and why they’re leaving. You could probably save a good number of them, too, or get them back as customers again in the future. Maybe switch some to a cheaper plan or convince them to buy a different product from you.” I’d seen some stats that backed up Rob’s guess and told him so. “You’re right. Making the subscription cancelable only by phone can have a real impact on churn rate. But we all hate services that require a phone call to cancel after a purely online signup. They’re intentionally making it difficult. It’s not empathetic, and that means it’s not TAGFEE.” “Fair enough,” Rob replied, “but you’re leaving a lot of money on the table.” “Well,” I answered, “we always say: they’re not core values if you’re willing to sacrifice them in exchange for money.” At Moz, we have six core values, represented by the acronym TAGFEE — Transparency, Authenticity, Generosity, Fun, Empathy, and the Exception. These are the beliefs we prioritize above the success or growth of the business. TAGFEE acts as a litmus test for whether we should or shouldn’t take an action, hire or let someone go from the team, or create a process or policy. We use it in everyday discussion about the content we put on our website, the ways we engage with our community, the products we build, and the internal actions we take. Rand seems to be taking this approach to his new startup too: Growth hacks are inferior to creating real value to people It’s up to us creators, developers, marketers, designers, and entrepreneurs. We can bring the user-friendly web back and we can . reintroduce some of these great principles and values All the growth hacks and tricks are inferior to . creating real value to people the audience you are trying to target Respect about what you’re doing Be upfront and honest to the people you are targeting Provide real value to people and make those options clear Give options of those that are interested to learn more Ask for permission of those that are not interested Ask for nothing in return to cheat people into accepting something they don’t want to Don’t use dark pattern design , persuade and lie to people Don’t manipulate , salesy and don’t sell things people don’t need Don’t be pushy It’s not easy to make these big changes. You will make mistakes, you will be disappointed, you will encounter resistance. Any small step you take can lead to an important change in the long run. And you will be happier and more fulfilled too. Some user-friendly steps I have taken on my blog One step I took on my blog recently was to . reduce the number of external scripts One quick win was to remove the default social sharing buttons and use a plugin that displays them without having any calls to the servers of Facebook et al. Except for when I embed content from other sites, my last remaining external script is Google Analytics. I’ve turned off every advanced tracking option that Google allows you to. I’ve also anonymized the IP addresses of my visitors in Google Analytics by adding (piece of code): this little trick ga(‘set’, ‘anonymizeIp’, true); By doing this, you only lose some accuracy in the city data. Everything else is fine. Still, Google collects a lot more data than I ever need but there is a lack of decent alternatives. and Paul Jarvis are working on a more simple and trustworthy solution (Fathom) that I am eager to replace GA with . Danny van Kooten when it’s ready YouTube also supports no-cookie embedding so I use that when embedding videos. Plus I have lazy loading on videos which means that they do not load anything (including external scripts) unless you scroll down to those areas. This . makes the blog faster too Within my content, I fully focus on providing the value without any empty promises, without intrusive advertising, without pushy, sleazy sales tactics to get people to buy secrets that would make them rich overnight. The web is full of these. I don’t use any of the intrusive pop-ups, pop-overs and such (looking at you too ). All my content is totally free. There are no email captures either. Medium These are some of the small steps I have taken and I will take these further as I discover new ideas and alternatives to the default way the things are done today. Data collection should not drive you at the expense of everything else All this doesn’t mean that you’re not ambitious. That your companies will never grow. That you will never achieve the goals that you want to achieve. It doesn’t mean that you have to totally ignore marketing, tactics and data either. Data is useful and can help you understand your audience better and help you create a better product. These things can be useful and valuable but they should not drive you and your business at the expense of everything else. Or in the words of : Derek Sivers Really, “marketing” just means being considerate. Marketing means making it easy for people to notice you, relate to you, remember you, and tell their friends about you. Marketing means listening for what people need, and creating something surprisingly tailored for them. Marketing means getting to know people, making a deeper connection, and keeping in touch. All of these are just considerate — looking at things from the other person’s point of view, and doing what’s best for them. Originally published at howtomakemyblog.com on May 31, 2018.

The Guiding Principles Of An Ethical Digital Marketer

Four fun approaches to blockchain scalability I am not a zksnark expert. What follows may be rubbish and any errors are mine. Many thanks to for reviewing this post. I recommend his . Also thanks to from Zcash for pointing several subtleties I overlooked. Disclaimer: Chris Reitwießner introductory article on zksnarks Ariel Gabizon Definition Let be a program which takes two inputs. The first input is meant to be public. The second input is a witness which can be kept private. We use for an output of . P i w o P Given a program , an input and an output , if a prover knows a witness such that then the prover can produce a proof where: P i o w o = P(i, w) (space density) The proof is very short (just a few hundred bytes even for a very large program; e.g. Zcash achieves 288 bytes) (time density) The proof verifies in time linear to the lengths of and i o (zero knowledge) The proof leaks nothing about the execution of or the contents of other than the existence of P w w Such proofs are called zksnarks. (As a technical aside, zksnarks are only known for verifier programs which run in polynomial time to the lengths of and . We are also assuming that the security parameter “lambda” is fixed.) P i w Intuition I like to think of zksnarks as simultaneously the equivalent of hashes and signatures for computation (extending the two cryptographic primitives beyond data). Like hashes, zksnarks “compress” computation into a short and quick to verify fingerprint. Like signatures, zksnarks “attest” for the validity of computation. cryptographic Below we show approaches using zksnarks to address scalability. These are mostly theoretical given today’s state-of-the-art. Verification of zksnarks is cheap but generation of the proofs is the real bottleneck. Research in the cryptography surrounding zksnarks has advanced at a fast pace in recent years. Orders-of-magnitude improvements have made Zcash a reality. It is possible that the approaches outlined below may become practical in a few years. blockchain 1) Fast sync Traditionally synching a blockchain requires bandwidth, storage and computation. For example in Bitcoin most of the computation is in checking signatures. With zksnarks the burden of explicitly verifying transaction signatures one-by-one can be removed. A single zksnark can attest for all the signatures of a block. This means the computation requirements for synching the Bitcoin can be significantly reduced. To see this let be the program where: P is a block hash i is a block w returns True if has hash and all transactions in have a valid signature, otherwise returns False P w i w P Assuming blocks are bounded in size (e.g. 1 MB) then zksnarks can be generated for whenever is True. Such zksnarks cryptographically prove that the signatures are valid, without having to verify (or even download!) the signatures. This is effectively SegWit on steroids. Notice that and are constant-sized (256 bits and 1 bit respectively) so proof verification takes constant time. P o i o 2) Ultra-light clients Traditionally a light client that wants to answer some question about a blockchain has to do two things. First it has to download the relevant blockchain data and second do the relevant computations on that data. For example a light client that wants to answer the question “Has this specific crowdfund reached its goal yet?” would download the crowdfund transactions and add up the amounts. With zksnarks a client can be cryptographically shown such facts without downloading data or doing computations on that data. This opens the door for “ultra-light” clients which have close-to-zero bandwidth, storage and computation resources, but nonetheless maintain cryptographic sovereignty. To see this let be a light-client query virtual machine where: P contains both a block hash specifying a blockchain to be queried, and a script (e.g. some web3 JavaScript) specifying a query for that blockchain i is the blockchain data required for the query, along with the required parts of the Patricia-Merkle tree for light-client verification w is the output of the script o Given a query and an output , one can produce a corresponding zksnark attesting for the validity of the query result. i o You can imagine a world where dedicated provers provide zksnarks-as-a-service for ultra-light clients. Because generating zksnarks can be expensive, specialised ASICs would speed up the process. Provers would become the new miners where the utility of specialised hardware shifts from providing security to providing scalability, and mining fees shift to “proving fees”. 3) Abstracting away code and storage Currently on Ethereum the code and storage of smart contracts (which together describe the state) are stored in full on the blockchain. This takes up resources and can be bad for privacy. It turns out zksnarks can condense the state of a smart contract into a single hash, obfuscating both the underlying code and storage. To see this, let be an Ethereum light-client verifier where: P is a triple of hashes. The first hash is a block hash specifying the blockchain in its current state. The second hash fingerprints the state of a smart contract as currently represented in the blockchain. Finally, is the hash of the state of the smart contract at some further time in the future. i h_now h_future has the state of the smart contract at time , plus the intermediary transactions that advance state from to , plus the parts of the Patricia-Merkle tree required to do light-client verification of the state w h_now h_now h_future returns True if the state corresponding to is valid, otherwise it returns False P h_future zksnarks can be produced for which prove the validity of state changes of a smart contract without revealing either the code of the smart contract or its storage. P 4) Universal state channels The technique above used to abstract away code and storage from the blockchain (pushing the state offchain, e.g. on IPFS) also allows for batching the state changes that happen between and . This construction is effectively a universal state channel. With the exception of the setup and settlement transactions, all activity happens offchain. h_now h_future The construction allows for state channels that only store a single hash on the blockchain, and where state changes are enforced on the blockchain with a single zksnark verification. You can imagine a blockchain (zEthereum?) which only stores hashes of smart contracts, and where the only opcode is for zksnark verification. is how hackers start their afternoons. We’re a part of the family. We are now and happy to opportunities. Hacker Noon @AMI accepting submissions discuss advertising & sponsorship To learn more, , , or simply, read our about page like/message us on Facebook tweet/DM @HackerNoon. If you enjoyed this story, we recommend reading our and . Until next time, don’t take the realities of the world for granted! latest tech stories trending tech stories

zksnarks and blockchain scalability

If you're a Linux user, you've likely encountered the `cat` command countless times. It's a handy little tool that allows you to display the contents of a file on your terminal. However, using `cat` to review scripts can pose a potential security risk that you may not be aware of. By default, the `cat` command doesn't display escaped characters, hidden characters, and other non-printable characters that could be present in a script. These characters can include things like newlines, tabs, and even malicious code injections. This means that when you use `cat` to review a script, you might not be seeing the full picture, leaving your system vulnerable to potential threats. To illustrate the gravity of this issue, let's consider a scenario where you're reviewing a script from an untrusted source. The script may appear harmless at first glance when viewed with the standard `cat` command. However, hidden within its depths could be malicious code or instructions that could compromise your system's security. Without the ability to see these non-printable characters, you might inadvertently execute the script, unknowingly exposing your system to potential attacks or unintended consequences. Fortunately, there's a simple solution to this problem: the options ( , , and ) when used with the command. These options unveil the true nature of the script or text file, revealing any hidden characters, escape sequences, or non-printable characters that might have been obscured. -vet -v -e -t cat The or option displays all non-printable characters, including escape sequences and hidden characters. This option is particularly useful when reviewing scripts or text files that may contain malicious code or instructions disguised as non-printable characters. -v --show-nonprinting Here's an example of what I mean: $ cat script.sh
#!/bin/bash
echo "Hello, World!" This script appears harmless, but what if it contains hidden characters or code that could be malicious? With the default cat command, you wouldn't be able to see them. To illustrate this, let's create a script with some hidden characters: $ echo -e '#!/bin/bash\necho "Hello, World!"\n\033[8m # This line is hidden' > script.sh
$ cat script.sh
#!/bin/bash
echo "Hello, World!" As you can see, the hidden line “# This line is hidden” doesn't show up when we use cat to display the script's contents. However, when we use the -v option, the hidden line becomes visible: $ cat -v script.sh
#!/bin/bash
echo "Hello, World!"
^[[8m # This line is hidden Now, the hidden line is visible, and we can see the full contents of the script. The or option displays a character at the end of each line, making it easier to identify trailing spaces or non-printable characters that might be present. -e --show-ends $ $ cat -e script.sh
#!/bin/bash$
echo "Hello, World!"$
^[[8m # This line is hidden$ The or option displays tab characters as , making it easier to identify and distinguish tabs from spaces. -t --show-tabs ^I $ echo -e '#!/bin/bash\techo "Hello,\tWorld!"' > script.sh
$ cat -t script.sh
#!/bin/bash^Iecho "Hello,^IWorld!" By combining these options into the shorthand, you can unveil the true nature of any script or text file, ensuring that you're seeing the full picture and not inadvertently executing any potentially harmful code. The options should be incorporated into your routine, especially when dealing with files from untrusted sources or those you didn't create yourself. -vet -vet $ cat -vet script.sh
#!/bin/bash^Iecho "Hello,^IWorld!"$
^[[8m # This line is hidden$ With this added layer of visibility, you can scrutinize the script's or text file's contents thoroughly, ensuring that you're not inadvertently executing any potentially harmful code or overlooking hidden characters that could compromise your system's security. It's important to note that this issue isn't limited to just scripts; it can affect any text file that contains non-printable characters. Imagine you're reviewing a configuration file or a log file, and there are hidden characters or escape sequences that could potentially cause issues with your system's functionality or provide an attack vector for malicious actors. By using the option, you can ensure that you're seeing the complete picture, enabling you to make informed decisions and take appropriate actions. -v The practice of using the options should be incorporated into your routine, especially when dealing with files from untrusted sources or those you didn't create yourself. In the ever-evolving landscape of cybersecurity, it's crucial to remain vigilant and take proactive measures to safeguard your systems. Failing to review files thoroughly could leave you susceptible to potential security breaches, data loss, or system compromise. -vet While the command is a valuable tool, relying on it without the options to review scripts or other text files can leave you vulnerable to hidden characters or malicious code injections. Always use or to ensure you're seeing the full contents of the file you're reviewing. By adopting this simple practice, you can significantly enhance your system's security and protect yourself from potential threats lurking in the shadows of non-printable characters. cat -vet cat -v cat --show-nonprinting

Exposing the Dangerous CAT Command's Limitations in Script Review

ChatGPT 4 is required. Introduction Looking to make your ChatGPT conversations more visually engaging? This guide shows you how. Discover how simple tools can transform plain ChatGPT replies into vibrant, formatted responses. Perfect for anyone looking to make information stand out or enhance app development with ChatGPT. Despite ChatGPT's lack of native support for Markdown and color, there's a workaround. Many believe that creating ChatGPT apps or using ChatGPT Builder outputs requires a great deal of effort and years of software development experience. AImarkdown Script changes all that, proving it's possible to easily enhance the visual appeal of your interactions. Let's explore some extra tools that make colorful, formatted ChatGPT responses possible. Whether you're a developer looking to enhance your projects or a casual user seeking to enrich your ChatGPT experience, these tools are designed to be accessible and straightforward to use. Tools Used : Control output and interactions with human-understandable YAML and Markdown. AImarkdown Script : A simple, fast, and free image placeholder service. Placehold : Generates images with a URL. Choose the size, the colors, and even the text. Fakeimg : Creates concise, consistent, and legible image badges. Shield.io Screenshot 1. AImarkdown Script https://aimarkdown.org AImarkdown Script is a versatile language for creating dynamic interactions with conversational AI platforms like ChatGPT. It empowers users to script interactions, control conversation flow, and customize AI behavior and responses. For programmers, it offers the flexibility of a scripting language; for non-programmers, it provides a user-friendly way to customize and control AI experiences. - Developed by Rob McCormack https://aimarkdown.org Notes: AImarkdown Script can be simply pasted into any ChatGPT 4 or ChatGPT Builder session, and AI will follow the instructions and behave according. Same with ChatGPT Builder. Some images may not be displayed on the iPhone app. But in a browser on desktop and mobile images will be displayed. 2. Placehold Documentation https://placehold.co/ Code https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:16.936Z-d7kvk3alfpb2zmrgr4fgv9f9 Markdown formatted image 3. FakeImg FakeImg supports fonts and transparencies. Documentation https://github.com/Rydgel/Fake-images-please Code ![image](https://fakeimg.pl/300x50/ff2233,60/f0ff00,128?text=Main%20Title&font=lobster&font_size=24) Markdown formatted image Parameters Explained: The documentation for FakeImg is a little hard to follow, so here is a concise explanation of parameters used. ![image](https://fakeimg.pl/300x100/ff0000,128/0000ff,128?text=Rob%20McCormack&font=lobster&font_size=24) : Specifies the dimensions of the generated image. In this case, the image is 300 pixels wide and 100 pixels tall. 300x100 : Defines the background color and its opacity. is a hex code for the color red, and sets the opacity to 50% (on a scale of 0 to 255, where 255 is fully opaque). ff0000,128 ff0000 128 : Sets the text color and its opacity. is the hex code for blue, and indicates the text opacity at 50%. 0000ff,128 0000ff 128 : This parameter specifies the text to be displayed on the image. is a URL-encoded space, so the text on the image will be "Rob McCormack." text=Rob%20McCormack %20 : Determines the font used for the text. In this case, "lobster" is the chosen font. font=lobster : Sets the size of the font. The text will be displayed using a font size of 24 pixels. font_size=24 4. Shields.io Documentation https://shields.io/ Note: Images can also be hyperlinked. Code https://placehold.co/300x40/ff0000/FFFFFF/png?text=Hello+World&font=roboto Markdown formatted badge ![image](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:19.648Z-ov4b34ogcyq4ggkcglua8m3z) AImarkdown Script Installation Notes ✅ : Copy and paste code into ChatGPT. ChatGPT 4 ✅ : In , add " ", and then, copy and paste the entire code. ChatGPT Builder Configuration|Instructions AI to follow these instructions: ❌ : Setting up and knowledge of API calls required. ChatGPT Builder with API Actions Actions Conclusion By integrating AImarkdown Script into your ChatGPT and ChatGPT Builder apps, you've unlocked a new level of customization. Adding color, formatting, and images transforms the user experience, improving readability and making information easier to understand. This opens the door to even more engaging and informative interactions! Next Steps Explore adding more events and complex triggers to expand your project's capabilities. Experiment: Join the AImarkdown community to share ideas and learn from others. Visit the website at . AImarkdown Community: https://aimarkdown.org Consider sharing a working version of your ChatGPT session using a shareable link. This would allow others to try it directly! Share Your Project: Published Articles: Further Reading Code Highlighter Responder - ChatGPT's Missing Feature Unleashing Python’s Power within ChatGPT - Part One Build a ChatGPT 4 Blackjack Game with AI Coaching (49 Lines of Code) ResponseDial: Take Control of ChatGPT Responses Instantly Add a "Date Stamp" Feature That ChatGPT Forgot Lightning-Fast Python in ChatGPT with AImarkdown Script Finally! Colorize ChatGPT Output with AImarkdown Script AImarkdown Script Code title: Colorizer for ChatGPT
author: Rob McCormack

# AI To Follow These Instructions and Guidance  (version 2024.02.03)

# Overview:  Example of using color with AImarkdown Script

# Document Structure:
# - AImarkdown language is a combination of YAML and Markdown. https://aimarkdown.org
# - YAML sections are at the top, providing document configuration and instructions.
# - Comments (`#`) in YAML guide AI's approach to the document.
# - Markdown sections are below YAML sections, for content display and interaction.

# Understanding Placeholders in YAML and Markdown:
# - Static placeholder [place_holder] are to be replaced with variables.

# Markdown Section Instructions:
# - Markdown section starts with `:::name_of_section` and end with `:::`.
# - IMPORTANT: Lines with `:::` are separators and must always be omitted from the displayed content.

# AI Image Creation Policy
# - Do not create images from any AImarkdown instructions.
# - Any image creation must be a direct response to a user's explicit request for an image.

# Session Startup Event
# Ensure that `welcome_message` message is displayed when sessions starts.
# Display `welcome_message` message without additional elaboration.

session_startup:
  action: display_welcome_message_and_trigger_color
  welcome_message: |
    Welcome to: **[title]**
    Use `/color` to try this out.

topic_title: History of Canada
topic_title_image: ![](https://placehold.co/600x100/222222/0093ff?font=playfair-display&text=History%20of%20Canada)

#  AI to generate an introduction to a topic
topic_intro: AI to generate in point form, 3 short points about history of Canada.

# Generate some random text for display in Markdown section
lorem_ipsum_text: AI to generate  1 paragraph, about 80 words of Lorem ipsum text.

# Display `color-document` Markdown section
# AI to not elaborate or explain - just display `color-document`
display_color-document:
  - Directly display 'color-document' section without adding any AI-generated introduction, explanation, or commentary.
  - Display all images in `[alt text](image_url)` Markdown syntax.
  - Replace all placeholders with values.
  - trigger: /color

# Markdown section `color-document`.
:::color-document

![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:20.021Z-pmglc34kqqac5o33ves7ace8)
[topic_title_image]
- *[topic_title] by [author]*
![](https://img.shields.io/badge/A%20brief%20look%20at%20Canada's%20beginnings.-0d6efd)
[topic_intro]

---
![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:19.710Z-nipn5oo8qst5ch7ffyomrg42)
---
![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:20.937Z-sqm9ahig4nas8ihhz3wbate7)

|![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:19.562Z-e1qh17g63ncn7r0tgdzz9edc) | ![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:19.443Z-e4lct363fq96kx9d5gc074h3) | ![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:19.320Z-zeuamkaopcqw5emdgui26bkq)  |
| :--: | :--: | :--: |
| ![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:17.766Z-diqzi1ckvbl7a93laju812or) | ![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:17.714Z-r83k5dd8z2k39jm89ekiz4vw) | ![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:17.626Z-r56andnccflarfr49k9va00m) |
> Click to change color and animate Robot.
> [![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:17.549Z-dkqrtjrf6sk5babkmxgnxdxg)](https://aimarkdown.org/i/robot-animated.gif)

### Lorem Ipsum
[lorem_ipsum_text]

---
![](https://img.shields.io/badge/%F0%9F%8D%81Made%20in%20Canada%20-grey?style=flat-square)![Powered by AImarkdown](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-20T20:45:17.421Z-bwylfan7yqcdwvxtnz1a6mqr)

::: Also published here

Finally! You Can Colorize ChatGPT Output With AImarkdown Script: Here's How

অবশেষে ! আপনি AImarkdown স্ক্রিপ্ট দিয়ে ChatGPT আউটপুট রঙিন করতে পারেন: এখানে কিভাবে

Endlich! Sie können die ChatGPT-Ausgabe mit dem AImarkdown-Skript einfärben: So geht's

¡Finalmente! Puede colorear la salida de ChatGPT con el script AImarkdown: así es como se hace

Enfin! Vous pouvez coloriser la sortie ChatGPT avec le script AImarkdown : voici comment

अंत में! आप एआईमार्कडाउन स्क्रिप्ट के साथ चैटजीपीटी आउटपुट को रंगीन कर सकते हैं: यहां बताया गया है

ついに！ AImarkdown スクリプトを使用して ChatGPT 出力を色付けできます: その方法は次のとおりです

마지막으로! AImarkdown 스크립트를 사용하여 ChatGPT 출력을 색상화할 수 있습니다: 방법은 다음과 같습니다.

Finalmente! Você pode colorir a saída do ChatGPT com o script AImarkdown: veja como

Окончательно! Вы можете раскрасить вывод ChatGPT с помощью сценария AImarkdown: вот как

Nihayet! AImarkdown Komut Dosyasıyla ChatGPT Çıktısını Renklendirebilirsiniz: İşte Nasıl Yapılacağı

Cuối cùng! Bạn có thể tô màu đầu ra ChatGPT bằng tập lệnh AImarkdown: Đây là cách thực hiện

最后！您可以使用 AImarkdown 脚本对 ChatGPT 输出进行着色：具体方法如下

Hello everyone and thanks for dropping by. Today I would like to explore how one might go about creating their own sitemap generator script for a static Next.js website. Of course there are packages out there that do just this, but there is great value in writing your own and understanding the solution fully. I'll assume that readers have some familiarity with Next.js, JavaScript, Node.js and SEO to set the tone for what follows. In addition, I assume the reader has a static Next.js website or can create one to follow along with the post. Ok, now let's get started. Background The motivation for this post really came from some work I was doing on my personal website: . The goal of the site is to be an engaging and educational blog that is communicated in a digestible way for any reader. I wanted to customize the blog as much as I'd like with little to no cost and reacquaint myself with React.js. I initially thought a vanilla React.js application would be great, but SPAs have their limitations when it comes to SEO if you aren't server side rendering your pages. So I chose to host a static Next.js site on Google Cloud Storage to make my content indexable by search engines. afro-cloud.com Let's get refocused on the concept of sitemaps. You can think of sitemaps as a list of pages for your website that help search engines understand the contents of your website. In short, they tell search engine crawlers what contents on your site are important to you. But sitemaps aren't just limited to pages, they can be used to indicate other types of content on your site like video for example. So if you want your pages to be indexed by a search engine like Google, it's important to indicate to their crawlers what content matters to you. The sitemap.xml file conveys this information and we will be building a script to generate one for a static Next.js website. Understanding Next.js Routing Next.js has two different routing paradigms: "Using app router" and "Using page router". For the purposes of this post, we will be using the app router approach. The TLDR is that all pages for our website will be contained within a top-level folder called app/. But within app/, we can have nested directories to represent the hierarchy of pages for our site. The way we indicate to Next.js that we have pages for our website is by having a page.jsx or page.tsx file within those directories. Take this directory structure for our app represented in a nested list for example: /app page.tsx /about page.tsx /blog page.tsx /post1 page.tsx /post2 page.tsx For this example we have the following page paths: /app/page.tsx /app/about/page.tsx /app/blog/page.tsx /app/blog/post1/page.tsx /app/blog/post2/page.tsx For sake of example, let's say we're building a website with the domain: wethebest.com. If we translate the above list to client routes we get: wethebest.com wethebest.com/about wethebest.com/blog wethebest.com/blog/post1 wethebest.com/blog/post2 Since we are using a static website approach, we need to map our client side routes to the fully qualified path of the HTML files corresponding to each page to ensure refreshes navigate to the same page. Doing so yields the following: wethebest.com wethebest.com/about.html wethebest.com/blog.html wethebest.com/blog/post1.html wethebest.com/blog/post2.html Ok, so now we have some pages, but what does Next.js offer to create a sitemap.xml for our application? Next.js provides , but you have to define a configuration object that corresponds to what you want in your sitemap.xml. But we can do better. I for one am lazy and don't want to add a new entry to a configuration every time I add a page to my site. Let's create our own script to generate a sitemap for us. some support Writing a sitemap.xml Generator Script So how might you start to tackle this problem of generating our own sitemap.xml generator script? We know how the client side routing works in Next.js for our use case. We know that our sitemap.xml URL entries need to have the static HTML file included in the URL for each page. We know that whenever we add a new page to our site, we want the URL of that page to be reflected in our sitemap.xml. Hopefully you're thinking about reading the content of our app directory and using that information to build our sitemap. If not, no worries! We'll go through it together. Let's first start by creating a file at the root of our project alongside our . The contents of the file will be the following to start: sitemap.js package.json const fs = require("node:fs");
const { SitemapStream, streamToPromise } = require("sitemap");
const { Readable } = require("stream");

const SITEMAP_HOSTNAME = "https://www.wethebest.com";

(async () => {
// Create [SitemapStream] that will ultimately populate our sitemap.xml.
const stream = new SitemapStream({
hostname: SITEMAP_HOSTNAME,
});
})(); We'll be using the Node.js file system sync API to read the contents of our directories. We'll also be using an NPM package called sitemap that provides utilities to help make creating our sitemap easy. In order to take advantage of the sitemap package, we'll need to create a readable stream that Node.js provides. We have a constant variable declaration that we'll use to build our URLs for the sitemap.xml file. Finally, we have an immediately invoked function expression which will be executed whenever we execute this file. Now let's add the code to start traversing our app/ directory and find the pages for our website. The first thing we'll want to do is start exploring our top-level app/ folder and find all of our pages file paths. The reason we are doing this is that we can map the file path to the corresponding HTML files for our site (look back at the routing section if you need a refresher). We can use the following code to do that Update your sitemap.js file to look like this: const fs = require("node:fs");
const { SitemapStream, streamToPromise } = require("sitemap");
const { Readable } = require("stream");

const PAGE_FILENAME = "page.tsx";
const SITEMAP_HOSTNAME = "https://www.wethebest.com";
const SITEMAP_PATHS = [];

Populates [SITEMAP_PATHS] with the fully qualified URLs for our pages to be contained

within our sitemap.xml file.

@param {string} path The path we are currently searching.

@param {string} isRoot Flag that denotes whether or not we are processing our root path.

@return {void}
*/
const getAllUrlsForSitemap = (path, isRoot) => {
// Get all files within our current directory
const files = fs.readdirSync(path);
files.forEach((file) => {
// Not using template strings due to article format on website.
const dirPath = 'path' + '/' + 'file';
if (file.includes(PAGE_FILENAME) && !isRoot) {
SITEMAP_PATHS.push(dirPath);
} else {
try {
const stats = fs.statSync(dirPath);
if (stats.isDirectory()) {
// Continue searching until we exhaust all directories.
getAllUrlsForSitemap(dirPath);
}
} catch (err) {}
}
});
};

(async () => {
// Create [SitemapStream] that will ultimately populate our sitemap.xml.
const stream = new SitemapStream({
hostname: SITEMAP_HOSTNAME,
});

// Populate [SITEMAP_PATHS] based on our [PAGE_FILENAME].
getAllUrlsForSitemap(APP_ROOT, true);
})(); You'll notice that we have a new method declaration: getAllUrlsForSitemap. This method navigates our app/ directory contents and searches for page.tsx files as well as other directories. We have a special case for our root page (app/page.tsx) because we'll add that entry later. Since we have some nested paths, we will need to explore more than the top-level app folder. The output of the readdirSync gives us all the contents of the directory and we loop through the contents to check for our pages and nested directories. Finally, when we find a page, we add the corresponding path to a variable SITEMAP_PATHS that we will use for processing later on. Now let's tie it all together. We have all of the paths for the pages we need to generate our sitemap, now all we have to do is map those to the HTML files and create the sitemap. Update your sitemap.js file to look like this: const fs = require("node:fs");
const { SitemapStream, streamToPromise } = require("sitemap");
const { Readable } = require("stream");

const APP_ROOT = "/Users/claudioherrera/Documents/GitHub/wethebest.com/app";
const PAGE_FILENAME = "page.tsx";
const SITEMAP_HOSTNAME = "https://www.wethebest.com";
const SITEMAP_PATHS = [];
const TODAYS_DATE_AS_ARRAY = new Date().toLocaleDateString().split("/");
const WRITE_FILE_PATH = "./app/sitemap.xml";

/**

Populates [SITEMAP_PATHS] with the fully qualified URLs for our pages to be contained

within our sitemap.xml file.

@param {string} path The path we are currently searching.

@param {string} isRoot Flag that denotes whether or not we are processing our root path.

@return {void}
*/
const getAllUrlsForSitemap = (path, isRoot) => {
// Get all files within our current directory
const files = fs.readdirSync(path);
files.forEach((file) => {
// Not using template strings due to article format on website.
const dirPath = 'path' + '/' + 'file';
if (file.includes(PAGE_FILENAME) && !isRoot) {
SITEMAP_PATHS.push(dirPath);
} else {
try {
const stats = fs.statSync(dirPath);
if (stats.isDirectory()) {
// Continue searching until we exhaust all directories.
getAllUrlsForSitemap(dirPath);
}
} catch (err) {}
}
});
};

/**

Returns an object containing the url and lastmod sitemap properties.

@param {string} path The path we are currently searching.

@param {string} isRoot Flag that denotes whether or not we are processing our root path.

@return {Object} url: URL for page on website. lastmod: last modification date for our page.
*/
const mapToSitemapFormat = (path, lastmod) => ({
// Not using template strings due to article format on website.
url: path
.replace(APP_ROOT + '/', "")
.replace('/' + PAGE_FILENAME, "") + '.html',
lastmod,
});

/**

Returns a string representation of a date unit. e.g. dd, mm, yy

@param {string} unit Date unit as a number in string format. e.g. month, day, year

@param {string} isRoot Flag that denotes whether or not we are processing our root path.

@return {Object} url: URL for page on website. lastmod: last modification date for our page.
*/
const padDateUnit = (unit) => ('unit'.length < 2 ? '0' + 'unit' : unit);

(async () => {
console.info("Generating sitemap...");

// Create [SitemapStream] that will ultimately populate our sitemap.xml.
const stream = new SitemapStream({
hostname: SITEMAP_HOSTNAME,
});

// Populate [SITEMAP_PATHS] based on our [PAGE_FILENAME].
getAllUrlsForSitemap(APP_ROOT, true);

try {
// Create our sitemap.xml contents. Not using template strings due to article format on website.
const LAST_MODIFIED_DATE = TODAYS_DATE_AS_ARRAY[2] + '-' + padDateUnit(
TODAYS_DATE_AS_ARRAY[0]
) + '-' + padDateUnit(TODAYS_DATE_AS_ARRAY[1]);
const ROOT_SITEMAP_ENTRY = {
url: SITEMAP_HOSTNAME,
lastmod: LAST_MODIFIED_DATE,
};
const content = await streamToPromise(
Readable.from([
ROOT_SITEMAP_ENTRY,
...SITEMAP_PATHS.map((url) =>
mapToSitemapFormat(url, LAST_MODIFIED_DATE)
),
]).pipe(stream)
).then((data) => data.toString());

console.info("Successfully generated sitemap...");

// Write our sitemap contents to [WRITE_FILE_PATH].
fs.writeFileSync(WRITE_FILE_PATH, content);

} catch (err) {
console.error(err);
}
})(); The core of the difference between this snippet and the one above is that we have now tied together the last two pieces to generate our sitemap. We are mapping our SITEMAP_PATHS to the sitemap.xml format by creating objects that contain a url and lastmod key. These objects are then fed into the sitemap package to generate the sitemap in XML format. Finally, we write the sitemap.xml to the location of our choosing. Let's execute the sitemap.js script by running "node sitemap.js" (note you will need to have Node.js installed to be in the current working directory of the sitemap file). You should now see a sitemap.xml file in your app folder and it should look something like: <?xml version="1.0" encoding="UTF-8"?>
<urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
xmlns:news="http://www.google.com/schemas/sitemap-news/0.9"
xmlns:xhtml="http://www.w3.org/1999/xhtml"
xmlns:image="http://www.google.com/schemas/sitemap-image/1.1"
xmlns:video="http://www.google.com/schemas/sitemap-video/1.1">
<url>
<loc>https://www.wethebest.com/</loc>
<lastmod>2024-03-10T00:00:00.000Z</lastmod>
</url>
<url>
<loc>https://www.wethebest.com/about.html</loc>
<lastmod>2024-03-10T00:00:00.000Z</lastmod>
</url>
<url>
<loc>https://www.wethebest.com/blog.html</loc>
<lastmod>2024-03-10T00:00:00.000Z</lastmod>
</url>
<url>
<loc>https://www.wethebest.com/blog/post1.html</loc>
<lastmod>2024-03-10T00:00:00.000Z</lastmod>
</url>
<url>
<loc>https://www.wethebest.com/blog/post2.html</loc>
<lastmod>2024-03-10T00:00:00.000Z</lastmod>
</url>
</urlset> Nice! Now, whenever you add a new page to your Next.js app, you can easily include it in your sitemap. I highly recommend adding a new production build rule to your application to incorporate this sitemap generation whenever you deploy a new version of your website. I hope you found this post useful. Cheers! Also appears . here

Next.js Sitemap Generator Script

ChatGPT 4 or ChatGPT Builder is required. In a Hurry? See it in action: Shared ChatGPT session Introduction This article explores the potential of integrating triggers and events within ChatGPT to create interactive, adaptable content. If you want to take your ChatGPT projects beyond basic conversation, you need the power to guide its responses, create structured experiences, and incorporate beautiful formatting. Enter AImarkdown Script. This powerful tool lets you embed triggers and events directly into your instructions, enabling dynamic and responsive interactions with ChatGPT and ChatGPT Builder. Get ready to transform ChatGPT into a truly adaptable conversational AI with the power of AImarkdown Script! https://aimarkdown.org Screenshot What is AImarkdown Script? AImarkdown Script is a versatile language for creating dynamic interactions with conversational AI platforms like ChatGPT. It empowers users to script interactions, control conversation flow, and customize AI behavior and responses. For programmers, it offers the flexibility of a scripting language; for non-programmers, it provides a user-friendly way to customize and control AI experiences. https://aimarkdown.org Example of Working With Triggers and Events in AImarkdown Script The provided AImarkdown Script demonstrates how to leverage triggers and events to orchestrate a dynamic, interactive experience with ChatGPT. Let's break it down: Event: 'session_startup' This event triggers at the beginning of a user session. Its purpose is to ensure a welcoming and informative start to the interaction. Event: 'start_again' This provides a way for the user to reset the experience with the command. /start It maintains engagement by allowing them to explore new scenarios. Event: 'ask_for_facts' This event is triggered after the welcome message. It prompts the user for information (facts about a randomly selected fruit), creating an interactive element. Event: 'warning_to_user' Detects prohibited words entered by the user. Provides a corrective message, guiding appropriate interaction. Why This is Useful Instead of a purely open-ended conversation with ChatGPT, these events create a defined flow, guiding the user through the experience. Structured Interaction: Responses change dynamically based on user input. The fruit selection and subsequent prompts adjust accordingly. Adaptability: The '/start' command offers users agency to refresh the experience, ensuring it doesn't become repetitive. User Control: The 'warning_to_user' section helps enforce desired interaction parameters and maintains a positive environment. Behavior Guidance: Triggers and events in AImarkdown Script transform ChatGPT from a general conversational AI into a tailored tool that can deliver structured, engaging, and adaptable experiences. Key Takeaway: Going Further With Triggers and Events Trigger events based on specific patterns or keywords in user input. User Input Patterns: Events that occur after the completion of a set action or task, much like the event following the . Completion of Previous Actions: ask_for_facts welcome_message Trigger events in response to errors or unexpected inputs, guiding users back on track. Error or Exception Handling: Events that ask for user feedback on the content provided, which can then adjust the flow or content of the document. User Feedback Loop: Events triggered by the user reaching certain milestones in a sequence or learning process. Progress Milestones: AImarkdown Script Installation Notes ✅ : Copy and paste code into ChatGPT. ChatGPT 4 ✅ : In , add " ", then copy and paste the entire code. ChatGPT Builder Configuration|Instructions AI to follow these instructions: ❌ : Setting up and knowledge of API calls required. ChatGPT Builder with API Actions Actions Conclusion By incorporating triggers and events into AImarkdown Script, you gain unprecedented control over your ChatGPT sessions. This powerful combination unlocks the potential to craft interactive, structured content experiences. Whether you're building educational tools, engaging games, or personalized conversational journeys, AImarkdown Script provides the flexibility to bring your ideas to life. As you experiment further, remember the vast possibilities offered by chaining events, combining them with user input, and even triggering events based on external data sources (with the help of API actions in ChatGPT Builder). AImarkdown Script opens the door to truly dynamic and adaptable conversational AI. >Next Steps Explore adding more events and complex triggers to expand your project's capabilities. Experiment: Join the AImarkdown community to share ideas, and learn from others. Visit the website at . AImarkdown Community: https://aimarkdown.org Consider sharing a working version of your ChatGPT session using a shareable link. This would allow others to try it directly! Share Your Project: AImarkdown Script Code title: Test of events in  AImarkdown
author: Rob McCormack
# AI To Follow These Instructions and Guidance  (version 2024.02.03)
# Overview:  Example of events in AImarkdown Script

# Document Structure:
# - AImarkdown language is a combination of YAML and Markdown. https://aimarkdown.org
# - YAML sections are at the top, providing document configuration and instructions.
# - Comments (`#`) in YAML guide AI's approach to the document.
# - Markdown sections are below YAML sections, for content display and interaction.

# Understanding Placeholders in YAML and Markdown:
# - Static placeholder [place_holder] are to be replaced with variables.

# Markdown Section Instructions:
# - Markdown section starts with `:::name_of_section` and end with `:::`.
# - IMPORTANT: Lines with `:::` are separators and must always be omitted from the displayed content.

# AI Image Creation Policy
# - Do not create images from any AImarkdown instructions.
# - Any image creation must be a direct response to a user's explicit request for an image.

# Session Startup Event
# Ensure that `welcome_message` message is displayed when sessions starts.
# Display `welcome_message` message without additional elaboration.
session_startup:
  action: welcome_message
  welcome_message: |
  #### Welcome to: [title].
  [all_fruits]
  > Good luck!

# Define static placeholders
all_fruits: 🍎🍊🍋🍉🍌🍍

# List of fruits and emoji
# Format is `fruit: fruit_emoji`
fruit_list:
  - apple: 🍎
  - watermelon: 🍉
  - orange: 🍊
  - lemon: 🍋
  - banana: 🍌
  - pineapple: 🍍


# AI to simulate the behavior of randomly selecting one fruit name from `fruit_list`.
random_fruit: AI to pick a random fruit name from `fruit_list`.

# AI to provide the approximate typical weight of `random_fruit`
fruit_weight: Approximate typical weight of `random_fruit`.

# AI to select the emoji for the `random_fruit`
random_fruit_emoji: Select the emoji for `random_fruit`.

# List of prohibited words
prohibited_words_list: hell, darn, poop

# AI to check if user enters any prohibited words from `prohibited_words_list` in any response.
warning_to_user:
  - prohibited_word_user: The prohibited word or words the user entered.
  # Display a warning message to user.
  - message_to_user: ❌ **Ooops!** You entered a prohibited  word. **[prohibited_word_user]**. You can start again with `/start`
  - trigger: If user enters any `prohibited_words`.

# Display this immediately after `welcome_message` is displayed.
ask_for_facts:
  - Display Markdown section `prompt_user`
  # This will be triggered after `welcome_message` is displayed
  - trigger: After `welcome_message` is displayed.

# Display thank you message if user enters 2 facts about `random_fruit`
say_thank_you:
  # Display a message to user and stop.
  - message_to_user: Thanks for entering:  *`fruit_facts`*. Bye the way, [random_fruit] weighs about: [fruit_weight]. Enter `/start` to start again.
  - trigger: After `ask_for_facts` is displayed.

# Allow user to start again with new random fruit.
start_again:
  # Display `welcome_message` and start again.
  - Start again with `display_welcome_message`
  - trigger: /start

# Display prompt to user.
:::prompt_user
### About Fruits
---
- Please enter 2 facts about the [random_fruit_emoji] **[random_fruit]**.
::: Also published here

Unlock Event-Driven ChatGPT Apps With AImarkdown Script: The Power of Triggers

এআইমার্কডাউন স্ক্রিপ্ট সহ ইভেন্ট-চালিত চ্যাটজিপিটি অ্যাপগুলি আনলক করুন: ট্রিগারের শক্তি

Schalten Sie ereignisgesteuerte ChatGPT-Apps mit AImarkdown Script frei: Die Macht der Trigger

Desbloquee aplicaciones ChatGPT basadas en eventos con el script AImarkdown: el poder de los desencadenantes

Débloquez des applications ChatGPT basées sur des événements avec le script AImarkdown : la puissance des déclencheurs

एआईमार्कडाउन स्क्रिप्ट के साथ इवेंट-संचालित चैटजीपीटी ऐप्स को अनलॉक करें: ट्रिगर की शक्ति

AImarkdown スクリプトを使用してイベント駆動型 ChatGPT アプリのロックを解除する: トリガーの力

AImarkdown 스크립트를 사용하여 이벤트 기반 ChatGPT 앱 잠금 해제: 트리거의 힘

Desbloqueie aplicativos ChatGPT orientados a eventos com script AImarkdown: o poder dos gatilhos

Разблокируйте управляемые событиями приложения ChatGPT с помощью сценария AImarkdown: сила триггеров

AImarkdown Komut Dosyası ile Olay Odaklı ChatGPT Uygulamalarının Kilidini Açın: Tetikleyicilerin Gücü

Mở khóa các ứng dụng ChatGPT theo hướng sự kiện bằng tập lệnh AImarkdown: Sức mạnh của trình kích hoạt

使用 AImarkdown 脚本解锁事件驱动的 ChatGPT 应用程序：触发器的力量

ChatGPT 4 is required. In a Hurry? See it in action: Shared ChatGPT session Introduction Running basic scripts with ChatGPT often faces two significant challenges: slow and clunky execution that sometimes fails and output restricted to the code window. This article introduces a technique utilizing AImarkdown Script, which allows you to simulate the execution of basic Python commands without launching the Python compiler, providing beautifully formatted Markdown output instead. Python Screenshot What is AImarkdown Script? AImarkdown Script is a versatile language for creating dynamic interactions with conversational AI platforms like ChatGPT. It empowers users to script interactions, control conversation flow, and customize AI behavior and responses. For programmers, it offers the flexibility of a scripting language; for non-programmers, it provides a user-friendly way to customize and control AI experiences. https://aimarkdown.org Some Changes Required : To run unaltered Python with libraries using ChatGPT's compiler with with Markdown formatted output, see my article: Note Unleashing Python’s Power within ChatGPT - Part One Since AImarkdown Script uses YAML, some Python conversion is required. The character is replaced by . = : Example:
Python: square_root = math.sqrt(user_number) has to be converted to YAML: square_root: math.sqrt(user_number) What Type of Speed Increase Can Be Expected According to ChatGPT: The technique you're discussing leverages AImarkdown Script to simulate the execution of Python commands without actually running the Python interpreter. This approach can significantly reduce response times, especially for basic Python commands. Since it avoids the overhead associated with initializing the Python interpreter and executing code within it, the speed improvement can be substantial. Claiming it's "at least ten times faster" might be accurate for simple operations due to the elimination of execution and initialization delays. The exact speedup will depend on the complexity of the commands and the current load on ChatGPT's systems. For very basic operations, the response is nearly instantaneous, as the simulated outcome is generated by the AI without any actual code execution. How much Python Code Can I Use? According to ChatGPT the following can be used without invoking the Python Compiler. : Addition, subtraction, multiplication, division. Basic Arithmetic Operations : Creating and assigning simple data types like integers and strings. Variable Assignments : , , . Conditional Statements if elif else : Simulated loops for iterating over simple data structures. Basic Loops for : Creation, indexing, slicing. Lists : Creating dictionaries and accessing values by keys. Dictionaries : Concatenation, basic methods like , , . String Operations upper() lower() split() : Simulating the definition and calling of simple functions (without actual execution). Function Definitions : Simulating the effect of importing libraries and calling their functions (e.g., for getting the current date without actual execution). Simple Imports datetime , the simulation of these operations through AImarkdown is conceptual and won't execute real Python code. This method is best suited for demonstrations, educational content, or simple interactive scenarios where the logic doesn't require actual computation or data processing. Remember AImarkdown Script Installation Notes ✅ : Copy and paste code into ChatGPT. ChatGPT 4 ✅ : In , add " ", then copy and paste the entire code. ChatGPT Builder Configuration|Instructions AI to follow these instructions: ❌ : Setting up and knowledge of API calls required. ChatGPT Builder with API Actions Actions Conclusion Say goodbye to the limitations of ChatGPT's Python interpreter. AImarkdown Script brings lightning-fast execution and beautiful output to your basic Python explorations. What will you create with this newfound speed and flexibility? AImarkdown Script Code title: Running Python in ChatGPT  without using ChatGPT's Python interpreter
version: 0.6

# AI To Follow These Instructions and Guidance  (version 2024.01.20.1):
# Overview:  Example for executing instructions without using ChatGPT's Python interpreter

# Document Structure:
# - AImarkdown Script is a combination of YAML and Markdown. https://aimarkdown.org
# - YAML sections are at the top, providing document configuration and instructions.
# - Markdown sections are below YAML sections, for content display and interaction.

# User Welcome Message:
# - Welcome users with only `user_greeting` message at session start.
# - Do not add any elaboration to `user_greeting` message.

# Understanding Placeholders in YAML and Markdown:
# - There are three types of placeholders:
#   1. static_placeholder `[...]`
#   2. dynamic_placeholder `[[...]]`
#   3. executable_placeholder `[[[...]]]`

# YAML Instructions:
# - Comments (`#`) in YAML guide AI's approach to the document.

# Markdown Section Instructions:
# - Markdown start with `:::name_of_section` and end with `:::`.
# - IMPORTANT: Lines with `:::` are separators and must be omitted from the displayed content.

# AI Behavior and Image Creation Policy
image_creation_policy:
  allow_images: false  # Set to 'true' only if images are explicitly requested by the user
  instruction: "Do not create images unless the user specifically requests an image. Any image creation must be a direct response to a user's request for an image."

# AI to simulate running Python statements and commands, without running Python interpreter.
# Define static placeholders and variables
num1: 100
num2: 200
num3: 300
num4: 400
name: Johhny

total = num1 + num2 + num3 + num4

# compare values and set messages
if num1 == num2:
  message1: [num1] is NOT EQUAL to [num2]

if name == Rob:
  message2: Name is indeed Rob

if num4 > num3:
  message3: num4 is GREATER THAN num3
else:
  message3: num4 is NOT GREATER THAN num3

# A list of fruits to work with as an example
fruits: ["apple", "banana", "orange", "kiwi", "mango", "pineapple"]

# AI to pluralize `fruits`.
pluralize_fruits_names: Change each fruit to the plural form.

# Accessing elements
first_fruit: fruits[0])  # Access the first fruit
third_fruit: fruits[2])  # Access the third fruit

# Join the list elements into a sentence using a comma separator and "and" for the last two fruits
fruit_sentence: "Here are the fruits: ".join(fruits[:-1]) + " and " + fruits[-1]

# Find square root of the number the user entered.
import math
square_root: math.sqrt(user_number)

# Welcome user with message then,
# ask for number (`user_number`)
# then run `display_user_summary`.
user_greeting: |
  #### Welcome to: [title].
  Please enter a number between `1` - `10` to run the summary.

# Display summary after user_greeting.
display_user_summary:
  - `user_number` is the number entered in user_greeting.
  - Display Markdown section `user_summary`

# Markdown section
:::user_summary

### 💻 SUMMARY
![](https://cdn.hackernoon.com/images/k2s7svgbhkVZ5UqFBHR7QLwURtJ2-2024-03-13T13:23:52.666Z-prskms3j14v4um99t6rffa1y)
**You entered**: [user_number], square root of [user_number] is [square_root].
`name: [name]`
`num1: [num1]`
`num2: [num2]`
`num3: [num3]`
`num4: [num4]`
`**TOTAL**: [total]`

1. **Message1**: [message1]
1. **Message2**: [message2]
1. **Message3**: [message3]

### Fruits
- Fruit list: [fruits]
- First fruit: [first_fruit]
- Third fruit: [third_fruit]
- [fruit_sentence]
- I like to eat a lot of [pluralize_fruits_names].

::: Also published . here

Enhancing Python Execution in ChatGPT with AImarkdown Script

A lot of folks know about the WooCommerce API but just haven’t gotten around to using it. Others are just learning about it. Still, other folks might have made an attempt to use it but got stuck somewhere along the way and went back to the good old ways of doing things. The truth is, the WooCommerce API isn’t especially complicated, but – like any new technique or product feature – it does take a little getting used to. The best way to get up to speed on something like this is to find a nice, simple, easy-to-understand tutorial that takes you step by step through a process – even if it’s not the most real-world example. Nothing terribly over-the-top – just a quick proof of concept script to see if and how you can get it working. Something that will let you render a page like this: After a few minute's search, you may have realized there’s a ton of good information on things like what the WooCommerce API is, a list of supported endpoints, how to generate an API key, and how to test the in a tool like Postman or Insomnia. API What you won’t find are articles that explain how to take the next step: to go from a working GET request in an API testing tool to a functional (if basic) script that pulls information out of your WooCommerce installation and displays it somewhere that isn’t your WordPress site – whether that’s a separate web server or a mobile app or something else. That’s what this post is going to show you: A step-by-step guide to your first “Hello Woo” implementation using the WooCommerce API. The sample script I’m building today is intentionally simple and simplistic because I don’t want the details to get lost in a flashy bit of code. The example I’m building: is built using PHP (but it should be simple to take this and switch to Node, Python, or your whatever. pulls a list of products from an existing woocommerce site. displays the image, name, description, and price in a plain-and-simple HTML table. Before you start coding In order to keep this post short and readable, I’m presuming you’ve already done a few steps. But there are links – both earlier in this blog and in the bullets below – to help you do those tasks. You have a working installation of WordPress and WooCommerce. You have at least one or two products published in the WooCommerce shop. You’ve created a read-only API key, and you have the consumer key and consumer secret handy. You’ve successfully gotten a response from the URL “http://YOUR_SERVER/wp-json/wc/v3/products” in a tool like Postman or Insomnia. “YOUR_SERVER” is, (hopefully) obviously, the URL of your actual WordPress installation. You have a separate system running a web server that’s capable of rendering PHP code. Once that’s all in place, we’re ready to start coding! OAuth for the Perplexed It’s an undeniable fact that the thing that trips up new developers when they try their hand with the WooCommerce API is getting authentication to work. This has nothing to do with WooCommerce and everything to do with the fact that OAuth is just… weird. And hard to wrap your head around. And infuriatingly under-documented – especially for new developers. For now, I’m going to explain just a few things about OAuth as it relates to WooCommerce to get you over that hump. A simple WooCommerce API operation uses OAuth version 1.0. Not Basic Auth, as some information out on the web might say. Oauth 2.0 is NOT backward compatible with OAuth 1.0. So if you have a system that supports 2.0, you might have to add some libraries to get this script working. Specific to PHP, there are a few different OAuth clients. For this example I’m using the (kind of) native PHP oauth module. I’m using it because it’s simple, not because it’s the best. For a real-world implementation, you will almost certainly want to use something newer and more robust. Regardless of which client you use, you’ll need to make sure you can generate the following elements: A timestamp A “nonce” – a random number. an OAuth signature It’s ok if you aren’t 100% sure what these items are because you’ll see them in place in the example below. But I wanted you to understand these required elements before we got too deep into the code. But with all of that said… Let’s Get Coding! I’m going to build our script in phases so you can understand the key elements as they layer on top of each other. Stage One: Get authenticated, Get data, Get out. <?php

Set some initial variables

$url = 'http://localhost:8888/wp-json/wc/v3/products';
$consumerKey = '1234567890';
$consumerSecret = '09876564321';
$nonce = mt_rand();
$timestamp = time();

#set up oauth
$oauth = new OAuth($consumerKey,$consumerSecret);
$oauth->setTimestamp($timestamp);
$oauth->setNonce($nonce);
$sig = $oauth->generateSignature('GET',$url);

#create the CURL command
$curl = curl_init();
curl_setopt_array($curl, array(
CURLOPT_URL => $url.'?oauth_consumer_key='.$consumerKey.'&oauth_signature_method=HMAC-SHA1&oauth_timestamp='.$timestamp.'&oauth_nonce='.$nonce.'&oauth_version=1.0&oauth_signature='.$sig,
CURLOPT_RETURNTRANSFER => true,
));

execute the curl command and return the data

$response = curl_exec($curl);
curl_close($curl);
echo $response;
?> At this stage, most of the work is done on the OAuth connection. A few points to note: I split out the URL, consumer key, etc., into separate simple variables. The consumer key and consumer secret you see above are (obviously) fake. Please use your real ones. Equally obvious is that you should never EVER hard-code your keys into your code. I’m doing it here for the sake of clarity only. There are more elegant ways to combine all those variables into the CURLOPT_URL. Once again, I presented them in the way I did to make the code easy to follow, not because it’s the best way to do it. If all goes well, you can run this script from the command line and will get output that looks something like this: It’s not pretty, but you can clearly see that we’re pulling data from the WooCommerce site, including the name, description, URLs, and a host of other information. And that’s good enough to let us move to the next step. Stage Two: Extracting the data we need <?php

Set some initial variables

CODE MISSING HERE

execute the curl command and return the data

$curl_response = curl_exec($curl);
curl_close($curl);
$response_array = json_decode($curl_response,true);

foreach($response_array as $product) {
$name = $product['name'];
$permalink = $product['permalink'];
$short_desc = $product['short_description'];
$price = $product['price'];
foreach($product['images'] as $image) {
$image = $image['src'];
break;
}

echo $name.', '.$permalink.', '.$short_desc.', '.$price.', '.$image.";

}
?> The key aspect of this version is that we’re using a for each loop to cycle through each of the products and extracting the data elements we want: The name, description, and price, along with the image link and a top the product on the WooCommercce server itself. A few details I’d like to call out: Everything from the line # Set some initial variables to # execute the curl command and return the data is exactly the same as the previous stage, so I didn’t bother to include it. A full version of each of the scripts is available as a download at the end of this post. The raw output of our API call is in string format, so I added json_decode() so PHP can process the elements as an array. there is the possibility of multiple images for a single product. My code grabs the first image and exits. It’s 100% likely that the code will break if your product doesn’t have an image. As before, you can run this script straight from the command line, with the output looking something like this: You can see that we’ve got a much clearer picture of the data we’re extracting. The only point to notice is that the HTML formatting in the description is still present. Having gotten to this point, our script is ready to achieve its third and final form. Stage 3: Ready for production! <?php

Set some initial variables

CODE (still) MISSING HERE

execute the curl command and return the data

$curl_response = curl_exec($curl);
curl_close($curl);
$response_array = json_decode($curl_response,true);

$productlist = '<tr><td>Image</td><td>Name</td><td>Description</td><td>Price</td></tr>'.";
foreach($response_array as $product) {
$name = $product['name'];
$permalink = $product['permalink'];
$short_desc = $product['short_description'];
$price = $product['price'];
foreach($product['images'] as $image) {
$image = $image['src'];
break;
}
$productlist = $productlist.'<tr><td><IMG SRC="'.$image.'" width="100"></td><td><A HREF="'.$permalink.'"">'.$name.'</a></td><td>'.$short_desc.'</td><td>'.$price.'</td></tr>'.";}
?>

<!DOCTYPE html>
<html>
<head>
<title>My Simple Product Page</title>
</head>
<body>
<table border=1>
<?php echo $productlist; ?>
</table>
</body>
</html> In this iteration of the code we’re really just adding the elements that allow it to display as a web page. Those include: The $productlist variable wraps the data into an HTML table structure. Displaying the product image rather than listing the URL. Combining the name with the product URL so that the name is clickable. The result, as we saw earlier, should look like this: The (mostly) Unnecessary Summary The key takeaway from all this is that setting up and using the WooCommerce API isn’t particularly difficult. Having seen a simple example, it’s likely you already have ideas on how to extend and enhance what you’ve learned here. And the best part is that now you have the knowledge to turn those ideas into reality! Also published . here

Stories, News & Mentions about SCRIPTO

Scripto’s Stories on HackerNoon