You'll be working on our collaborative writing and production platform for TV and film

Scripto

Architectural Designs

Of the power of the eleventh horn of Daniel's fourth Beast, to change times and laws

Welcome to other chapters of Let’s Understand Chrome V8 In the past articles, we talked a lot about the JavaScript compiler of V8. The parser, the scanner, and the bytecode, as well as their fundamentals, kernel code, and key structure, were all we covered. In the next articles, we will walk through the compiler workflow, and watch how V8 transforms the JavaScript code into bytecode step by step. Figure 1 shows the workflow, it starts from the JavaScript code you wrote, goes through the scanner and parser, and finally generates bytecodes. In this article, I use d8.exe instead of v8.exe, since d8.exe can print out to the terminal directly as well as the d8 is very light versus v8. Note: 1. Read JavaScript code Below is our test case： function ignition(s) {
    this.slogan=s;
	this.start=function(){eval('console.log(this.slogan);')}
}
worker = new ignition("here we go!");
worker.start(); Below is the Execute() that is responsible for executing the JavaScript code. Quite simply, it first reads the JavaScript code from a file, then compiles and executes the code. 1.  bool SourceGroup::Execute(Isolate* isolate) {
2.  //............omit..................
3.      // Use all other arguments as names of files to load and run.
4.      HandleScope handle_scope(isolate);
5.      Local<String> file_name =
6.          String::NewFromUtf8(isolate, arg, NewStringType::kNormal)
7.              .ToLocalChecked();
8.      Local<String> source = ReadFile(isolate, arg);
9.      if (source.IsEmpty()) {
10.        printf("Error reading '%s'\n", arg);
11.        base::OS::ExitProcess(1);
12.      }
13.      Shell::set_script_executed();
14.      if (!Shell::ExecuteString(isolate, source, file_name, Shell::kNoPrintResult,
15.                                Shell::kReportExceptions,
16.                                Shell::kProcessMessageQueue)) {
17.        success = false;
18.        break;
19.      }
20.    }
21.    return success;
22.  } In line 5, the NewFromUtf8 gets the file name which is our case. In line 8, the ReadFile gets file content which is our case actually. 1.  Local<String> Shell::ReadFile(Isolate* isolate, const char* name) {
2.  //only the most important parts...............................
3.    char* chars = static_cast<char*>(file->memory());
4.    Local<String> result;
5.    if (i::FLAG_use_external_strings && i::String::IsAscii(chars, size)) {
6.      String::ExternalOneByteStringResource* resource =
7.          new ExternalOwningOneByteStringResource(std::move(file));
8.      result = String::NewExternalOneByte(isolate, resource).ToLocalChecked();
9.    } else {
10.      result = String::NewFromUtf8(isolate, chars, NewStringType::kNormal, size)
11.                   .ToLocalChecked();
12.    }
13.    return result;
14.  } Line 8 and line 10 return the JavaScript code depending on the type is ExternalOneByte or UTF8. Here, our case is UTF8, regarding String::ExternalOneByteStringResource, we will talk about it in the future. Back to SourceGroup::Execute(), and step into Shell::ExecuteString at line 14, its source is given below: 1.  bool Shell::ExecuteString(Isolate* isolate, Local<String> source,
2.                      Local<Value> name, PrintResult print_result,
3.                      ReportExceptions report_exceptions,
4.                      ProcessMessageQueue process_message_queue) {
5.  	//omit............................
6.  bool success = true;
7.  {
8.    if (options.compile_options == ScriptCompiler::kConsumeCodeCache) {
9.  	//omit............................
10.       } else if (options.stress_background_compile) {
11.  	//omit............................
12.       } else {
13.         ScriptCompiler::Source script_source(source, origin);
14.         maybe_script = ScriptCompiler::Compile(context, &script_source,
15.                                                options.compile_options);
16.       }
17.       Local<Script> script;
18.       if (!maybe_script.ToLocal(&script)) {
19.         // Print errors that happened during compilation.
20.         if (report_exceptions) ReportException(isolate, &try_catch);
21.         return false;
22.       }
23.       if (options.code_cache_options ==
24.           ShellOptions::CodeCacheOptions::kProduceCache) {
25.  	//omit............................
26.       }
27.       maybe_result = script->Run(realm);//here is executing bytecode.
28.  }
29.  } In line 13, our JavaScript is wrapped into a variable script_source which includes the line and column offset. Since V8 only compiles the JavaScript function that is executing exactly not the full JavaScript code, the variable script_source help the compiler record the compilation information. In line 14, start to compile JavaScript. 2. Parser initialization In the workflow, the first part is the scanner, the second is the parser. Actually, the scanner is passive and the parser is active, this means that the parser takes out a token from the compilation cache actively, and the scanner is waked up by the parser once the cache miss, and to generate tokens and fill the cache. Below is CompileUnboundInternal() called by ScriptCompiler::Compile that at above line 14. 1.  MaybeLocal<UnboundScript> ScriptCompiler::CompileUnboundInternal(
2.      Isolate* v8_isolate, Source* source, CompileOptions options,
3.      NoCacheReason no_cache_reason) {
4.  //omit...............
5.    i::Handle<i::String> str = Utils::OpenHandle(*(source->source_string));
6.    i::Handle<i::SharedFunctionInfo> result;
7.    i::Compiler::ScriptDetails script_details = GetScriptDetails(
8.        isolate, source->resource_name, source->resource_line_offset,
9.        source->resource_column_offset, source->source_map_url,
10.        source->host_defined_options);
11.    i::MaybeHandle<i::SharedFunctionInfo> maybe_function_info =
12.        i::Compiler::GetSharedFunctionInfoForScript(
13.            isolate, str, script_details, source->resource_options, nullptr,
14.            script_data, options, no_cache_reason, i::NOT_NATIVES_CODE);
15.    if (options == kConsumeCodeCache) {
16.      source->cached_data->rejected = script_data->rejected();
17.    }
18.    delete script_data;
19.    has_pending_exception = !maybe_function_info.ToHandle(&result);
20.    RETURN_ON_FAILED_EXECUTION(UnboundScript);
21.    RETURN_ESCAPED(ToApiHandle<UnboundScript>(result));
22.  } The above function generates the UnboundInternal stuff, the line 11 tells us that the stuff is just a Sharedfunction actually. But, what is Bind? A Sharedfunction cannot be executed directly, V8 needs to match a context to the Sharedfunction, the “match” is just the bing. Let’s step into GetSharedFunctionInfoForScript. 1.  MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
2.      Isolate* isolate, Handle<String> source,
3.      const Compiler::ScriptDetails& script_details,
4.   .................) {
5.  //omit.........................
6.  		{
7.      maybe_result = compilation_cache->LookupScript(
8.          source, script_details.name_obj, script_details.line_offset,
9.          script_details.column_offset, origin_options, isolate->native_context(),
10.          language_mode);
11.    }
12.    if (maybe_result.is_null()) {
13.      ParseInfo parse_info(isolate);
14.      // No cache entry found compile the script.
15.      NewScript(isolate, &parse_info, source, script_details, origin_options,
16.                natives);
17.      // Compile the function and add it to the isolate cache.
18.      if (origin_options.IsModule()) parse_info.set_module();
19.      parse_info.set_extension(extension);
20.      parse_info.set_eager(compile_options == ScriptCompiler::kEagerCompile);
21.      parse_info.set_language_mode(
22.          stricter_language_mode(parse_info.language_mode(), language_mode));
23.      maybe_result = CompileToplevel(&parse_info, isolate, &is_compiled_scope);
24.      Handle<SharedFunctionInfo> result;
25.      if (extension == nullptr && maybe_result.ToHandle(&result)) {
26.        DCHECK(is_compiled_scope.is_compiled());
27.        compilation_cache->PutScript(source, isolate->native_context(),
28.                                     language_mode, result);
29.      } else if (maybe_result.is_null() && natives != EXTENSION_CODE) {
30.        isolate->ReportPendingMessages();
31.      }
32.    }
33.    return maybe_result;
34.  } Line 7 is looking up the compilation cache, that I mentioned in the last article. Line 13, the parse_info is a wrapper for Parser, like the variable script_source mentioned upfront. Line 15, the Parser_info initialization, code is given below: 1.  Handle<Script> NewScript(Isolate* isolate, ParseInfo* parse_info,
2.                           Handle<String> source,
3.                           Compiler::ScriptDetails script_details,
4.                           ScriptOriginOptions origin_options,
5.                           NativesFlag natives) {
6.    Handle<Script> script =
7.        parse_info->CreateScript(isolate, source, origin_options, natives);
8.    Handle<Object> script_name;
9.    if (script_details.name_obj.ToHandle(&script_name)) {
10.      script->set_name(*script_name);
11.      script->set_line_offset(script_details.line_offset);
12.      script->set_column_offset(script_details.column_offset);
13.    }
14.    Handle<Object> source_map_url;
15.    if (script_details.source_map_url.ToHandle(&source_map_url)) {
16.      script->set_source_mapping_url(*source_map_url);
17.    }
18.    Handle<FixedArray> host_defined_options;
19.    if (script_details.host_defined_options.ToHandle(&host_defined_options)) {
20.      script->set_host_defined_options(*host_defined_options);
21.    }
22.    return script;
23.  } From line 6 to line 12 wrap the JavaScript code that is our case into the variable script, then initialize the line_offset and column_offset. It is like the script_source really as I said. Back to Compiler::GetSharedFunctionInfoForScript(), and step into CompileToplevel() at line 23. 1.  MaybeHandle<SharedFunctionInfo> CompileToplevel(
2.      ParseInfo* parse_info, Isolate* isolate,
3.      IsCompiledScope* is_compiled_scope) {
4.  //omit.......................
5.    if (parse_info->literal() == nullptr &&
6.        !parsing::ParseProgram(parse_info, isolate)) {
7.      return MaybeHandle<SharedFunctionInfo>();
8.    }
9.  //omit........................
10.    MaybeHandle<SharedFunctionInfo> shared_info =
11.        GenerateUnoptimizedCodeForToplevel(
12.            isolate, parse_info, isolate->allocator(), is_compiled_scope);
13.    if (shared_info.is_null()) {
14.      FailWithPendingException(isolate, parse_info,
15.                               Compiler::ClearExceptionFlag::KEEP_EXCEPTION);
16.      return MaybeHandle<SharedFunctionInfo>();
17.    }
18.    FinalizeScriptCompilation(isolate, parse_info);
19.    return shared_info;
20.  } In line 5, the literal() returns the abstract syntax tree(AST), and V8 starts up the compiler to generate AST if is nullptr. The first time, the AST is null, and steps into ParseProgram. 1.  bool ParseProgram(ParseInfo* info, Isolate* isolate,
2.                    ReportErrorsAndStatisticsMode mode) {
3.  //omit............................
4.    Parser parser(info);
5.    FunctionLiteral* result = nullptr;
6.    result = parser.ParseProgram(isolate, info);
7.    info->set_literal(result);
8.    if (result) {
9.      info->set_language_mode(info->literal()->language_mode());
10.      if (info->is_eval()) {
11.        info->set_allow_eval_cache(parser.allow_eval_cache());
12.      }
13.    }
14.    if (mode == ReportErrorsAndStatisticsMode::kYes) {
15.  //omit.............................
16.    }
17.    return (result != nullptr);
18.  } In line 5, create the parser, namely, parser initialization. 1.  Parser::Parser(ParseInfo* info)
2.      : ParserBase<Parser>(info->zone(), &scanner_, info->stack_limit(),
3.                           info->extension(), info->GetOrCreateAstValueFactory(),
4.                           info->pending_error_handler(),
5.                           info->runtime_call_stats(), info->logger(),
6.                           info->script().is_null() ? -1 : info->script()->id(),
7.                           info->is_module(), true),
8.        info_(info),
9.        scanner_(info->character_stream(), info->is_module()),
10.        preparser_zone_(info->zone()->allocator(), ZONE_NAME),
11.        reusable_preparser_(nullptr),
12.        mode_(PARSE_EAGERLY),  // Lazy mode must be set explicitly.
13.        source_range_map_(info->source_range_map()),
14.        target_stack_(nullptr),
15.        total_preparse_skipped_(0),
16.        consumed_preparse_data_(info->consumed_preparse_data()),
17.        preparse_data_buffer_(),
18.        parameters_end_pos_(info->parameters_end_pos()) {
19.    bool can_compile_lazily = info->allow_lazy_compile() && !info->is_eager();
20.    set_default_eager_compile_hint(can_compile_lazily
21.                                       ? FunctionLiteral::kShouldLazyCompile
22.                                       : FunctionLiteral::kShouldEagerCompile);
23.    allow_lazy_ = info->allow_lazy_compile() && info->allow_lazy_parsing() &&
24.                  info->extension() == nullptr && can_compile_lazily;
25.    set_allow_natives(info->allow_natives_syntax());
26.    set_allow_harmony_dynamic_import(info->allow_harmony_dynamic_import());
27.    set_allow_harmony_import_meta(info->allow_harmony_import_meta());
28.    set_allow_harmony_nullish(info->allow_harmony_nullish());
29.    set_allow_harmony_optional_chaining(info->allow_harmony_optional_chaining());
30.    set_allow_harmony_private_methods(info->allow_harmony_private_methods());
31.    for (int feature = 0; feature < v8::Isolate::kUseCounterFeatureCount;
32.         ++feature) {
33.      use_counts_[feature] = 0;
34.    }
35.  } From line 8 to line 18, take out the compilation information, such as scanner, and compilation mode. From line 9 to line 23, the important stuff is can_compile_lazily. Line 25 enables the native commands that start with %. Summary V8 uses UTF16 to encode JavaScript source code; V8 uses v8::internal::source to manage our JavaScript code; First, look up the compilation cache, then start up the compiler if the cache miss; The scanner is passive and the parser is active. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : WeChat Email v8blink@outlook.com Also Published here

Understanding Chrome V8 - Chapter 20: How Compilers and Parsers Work

Welcome to other chapters of Let’s Understand Chrome V8 For performance reasons, V8 uses the compilation cache to hold the SharedFunction which is the compiler result, the cache returns the result directly once the same JavaScript code is compiled again. You may ask why the same code is compiled again. Compile first, then execute — this is the pipeline of V8. Even the exact same code, the same function, still needs to be sent to the compiler, but not always recompiled. Actually, JavaScript code may change during execution, so the first action always is that we get the JavaScript code’s hash-value from the compilation cache, return the SharedFunction if the cache is hit, and recompile the JavaScript conversely. 1. Initialization and Filling The Isolate, which is the V8 virtual machine instance, is responsible for initializing the CompilationCache, isolate::Init as shown below: bool Isolate::Init(ReadOnlyDeserializer* read_only_deserializer,
                   StartupDeserializer* startup_deserializer) {
//omit...................
#define ASSIGN_ELEMENT(CamelName, hacker_name)                  \
  isolate_addresses_[IsolateAddressId::k##CamelName##Address] = \
      reinterpret_cast<Address>(hacker_name##_address());
  FOR_EACH_ISOLATE_ADDRESS_NAME(ASSIGN_ELEMENT)
#undef ASSIGN_ELEMENT

  compilation_cache_ = new CompilationCache(this); The last line is now a CompilationCache, namely, initialization. Let’s dive into it in more depth. 1.  class V8_EXPORT_PRIVATE CompilationCache {
2.   public:
3.    MaybeHandle<SharedFunctionInfo> LookupScript(
4.        Handle<String> source, MaybeHandle<Object> name, int line_offset,
5.        int column_offset, ScriptOriginOptions resource_options,
6.        Handle<Context> native_context, LanguageMode language_mode);
7.    InfoCellPair LookupEval(Handle<String> source,
8.                            Handle<SharedFunctionInfo> outer_info,
9.                            Handle<Context> context, LanguageMode language_mode,
10.                            int position);
11.    MaybeHandle<FixedArray> LookupRegExp(Handle<String> source,
12.                                         JSRegExp::Flags flags);
13.    void PutScript(Handle<String> source, Handle<Context> native_context,
14.                   LanguageMode language_mode,
15.                   Handle<SharedFunctionInfo> function_info);
16.    void PutEval(Handle<String> source, Handle<SharedFunctionInfo> outer_info,
17.                 Handle<Context> context,
18.                 Handle<SharedFunctionInfo> function_info,
19.                 Handle<FeedbackCell> feedback_cell, int position);
20.    void PutRegExp(Handle<String> source, JSRegExp::Flags flags,
21.                   Handle<FixedArray> data);
22.    void Clear();
23.    void Remove(Handle<SharedFunctionInfo> function_info);
24.    void Iterate(RootVisitor* v);
25.    void MarkCompactPrologue();
26.    void Enable();
27.    void Disable();
28.   private:
29.    explicit CompilationCache(Isolate* isolate);
30.    ~CompilationCache() = default;
31.    base::HashMap* EagerOptimizingSet();
32.    static const int kSubCacheCount = 4;
33.    bool IsEnabled() const { return FLAG_compilation_cache && enabled_; }
34.    Isolate* isolate() const { return isolate_; }
35.    Isolate* isolate_;
36.    CompilationCacheScript script_;
37.    CompilationCacheEval eval_global_;
38.    CompilationCacheEval eval_contextual_;
39.    CompilationCacheRegExp reg_exp_;
40.    CompilationSubCache* subcaches_[kSubCacheCount];
41.    bool enabled_;
42.    friend class Isolate;
43.    DISALLOW_COPY_AND_ASSIGN(CompilationCache);
44.  }; There’re three fundamentals of the above function that are important: 1. LookupScript() at line 3. It is used to look up the cache to take out the SharedFunction that holds the bytecodes corresponding to your JavaScript code. If hit, skip compilation into the execution unit directly. 2. PutScript() at line 13. If LookupScript misses, PutScript() puts the hash corresponding to your JavaScript code and the SharedFunciton that the compiler generates into the cache before execution. 3. LookupEval and PutEval at lines 7 and 16. They are the same as LookupScript and PutScript but specialize in the JavaScript eval method because the eval needs not only SharedFunction but also context. Let’s examine LookupScript in a little more depth: 1.  MaybeHandle<SharedFunctionInfo> CompilationCache::LookupScript(
2.      Handle<String> source, MaybeHandle<Object> name, int line_offset,
3.      int column_offset, ScriptOriginOptions resource_options,
4.      Handle<Context> native_context, LanguageMode language_mode) {
5.    if (!IsEnabled()) return MaybeHandle<SharedFunctionInfo>();
6.    return script_.Lookup(source, name, line_offset, column_offset,
7.                          resource_options, native_context, language_mode);
8.  }
9.  //..................separation..........................
10.  //..................separation..........................
11.  MaybeHandle<SharedFunctionInfo> CompilationCacheScript::Lookup(
12.      Handle<String> source, MaybeHandle<Object> name, int line_offset,
13.      int column_offset, ScriptOriginOptions resource_options,
14.      Handle<Context> native_context, LanguageMode language_mode) {
15.    MaybeHandle<SharedFunctionInfo> result;
16.    {
17.      HandleScope scope(isolate());
18.      const int generation = 0;
19.      DCHECK_EQ(generations(), 1);
20.      Handle<CompilationCacheTable> table = GetTable(generation);
21.      MaybeHandle<SharedFunctionInfo> probe = CompilationCacheTable::LookupScript(
22.          table, source, native_context, language_mode);
23.      Handle<SharedFunctionInfo> function_info;
24.      if (probe.ToHandle(&function_info)) {
25.        if (HasOrigin(function_info, name, line_offset, column_offset,
26.                      resource_options)) {
27.          result = scope.CloseAndEscape(function_info);
28.        }
29.      }
30.    }
31.    Handle<SharedFunctionInfo> function_info;
32.    if (result.ToHandle(&function_info)) {
33.  #ifdef DEBUG
34.      DCHECK(HasOrigin(function_info, name, line_offset, column_offset,
35.                       resource_options));
36.  #endif
37.      isolate()->counters()->compilation_cache_hits()->Increment();
38.      LOG(isolate(), CompilationCacheEvent("hit", "script", *function_info));
39.    } else {
40.      isolate()->counters()->compilation_cache_misses()->Increment();
41.    }
42.    return result;
43.  } We can find that the CompilationCache::LookupScript is only the entrance, and the key function, as well as the important code, are all in CompilationCacheScript::Lookup(). 1. In Lookup(), the argument source is the JavaScript that you as developers are actually writing, which is going to be compiled. 2. GetTable at line 20. It is the actual table that holds data into which PutScript put. 3. CompilationCacheTable::LookupScript at line 21. Look up the table for a result in which hit means a SharedFunction or miss is null. 4. In line 25: Check if the SharedFunction is the correct one depending on Origin. Let’s take a glance at HasOrigin. 1.  // We only re-use a cached function for some script source code if the
2.  // script originates from the same place. This is to avoid issues
3.  // when reporting errors, etc.
4.  bool CompilationCacheScript::HasOrigin(Handle<SharedFunctionInfo> function_info,
5.                                         MaybeHandle<Object> maybe_name,
6.                                         int line_offset, int column_offset,
7.                                         ScriptOriginOptions resource_options) {
8.    Handle<Script> script =
9.        Handle<Script>(Script::cast(function_info->script()), isolate());
10.    Handle<Object> name;
11.    if (!maybe_name.ToHandle(&name)) {
12.      return script->name().IsUndefined(isolate());
13.    }
14.    if (line_offset != script->line_offset()) return false;
15.    if (column_offset != script->column_offset()) return false;
16.    if (!name->IsString() || !script->name().IsString()) return false;
17.    if (resource_options.Flags() != script->origin_options().Flags())
18.      return false;
19.    return String::Equals(
20.        isolate(), Handle<String>::cast(name),
21.        Handle<String>(String::cast(script->name()), isolate()));
22.  } From line 1 to line 3, the comments provide the origin check rules. Specifically, the implementation is from line 14 to line 16. Next, let’s go into PutScript. 1.  void CompilationCache::PutScript(Handle<String> source,
2.                                   Handle<Context> native_context,
3.                                   LanguageMode language_mode,
4.                                   Handle<SharedFunctionInfo> function_info) {
5.    if (!IsEnabled()) return;
6.    LOG(isolate(), CompilationCacheEvent("put", "script", *function_info));
7.    script_.Put(source, native_context, language_mode, function_info);
8.  }
9.  //.......................separation............................
10.  void CompilationCacheScript::Put(Handle<String> source,
11.                                   Handle<Context> native_context,
12.                                   LanguageMode language_mode,
13.                                   Handle<SharedFunctionInfo> function_info) {
14.    HandleScope scope(isolate());
15.    Handle<CompilationCacheTable> table = GetFirstTable();
16.    SetFirstTable(CompilationCacheTable::PutScript(table, source, native_context,
17.                                                   language_mode, function_info));
18.  } As same as LookupScript(), the CompilationCache::PutScript is entrance, all important stuff is in CompilationCacheTable::PutScript. In PutScript, I would like to inform you that the cache table will be filling. Below is the CacheTable source code: 1.  // This cache is used in two different variants. For regexp caching, it simply
2.  // maps identifying info of the regexp to the cached regexp object. Scripts and
3.  // eval code only gets cached after a second probe for the code object. To do
4.  // so, on first "put" only a hash identifying the source is entered into the
5.  // cache, mapping it to a lifetime count of the hash. On each call to Age all
6.  // such lifetimes get reduced, and removed once they reach zero. If a second put
7.  // is called while such a hash is live in the cache, the hash gets replaced by
8.  // an actual cache entry. Age also removes stale live entries from the cache.
9.  // Such entries are identified by SharedFunctionInfos pointing to either the
10.  // recompilation stub, or to "old" code. This avoids memory leaks due to
11.  // premature caching of scripts and eval strings that are never needed later.
12.  class CompilationCacheTable
13.      : public HashTable<CompilationCacheTable, CompilationCacheShape> {
14.   public:
15.    NEVER_READ_ONLY_SPACE
16.    static MaybeHandle<SharedFunctionInfo> LookupScript(
17.        Handle<CompilationCacheTable> table, Handle<String> src,
18.        Handle<Context> native_context, LanguageMode language_mode);
19.    static InfoCellPair LookupEval(Handle<CompilationCacheTable> table,
20.                                   Handle<String> src,
21.                                   Handle<SharedFunctionInfo> shared,
22.                                   Handle<Context> native_context,
23.                                   LanguageMode language_mode, int position);
24.    Handle<Object> LookupRegExp(Handle<String> source, JSRegExp::Flags flags);
25.    static Handle<CompilationCacheTable> PutScript(
26.        Handle<CompilationCacheTable> cache, Handle<String> src,
27.        Handle<Context> native_context, LanguageMode language_mode,
28.        Handle<SharedFunctionInfo> value);
29.//omit......................
30.    }; From line 1 to line 11, the comments provide more corresponding explanations enough. Let’s go back to CompilationCacheScript::Put() and ahead to line 16 which is CompilationCacheTable::PutScript(). 1.  Handle<CompilationCacheTable> CompilationCacheTable::PutScript(
2.      Handle<CompilationCacheTable> cache, Handle<String> src,
3.      Handle<Context> native_context, LanguageMode language_mode,
4.      Handle<SharedFunctionInfo> value) {
5.    Isolate* isolate = native_context->GetIsolate();
6.    Handle<SharedFunctionInfo> shared(native_context->empty_function().shared(),
7.                                      isolate);
8.    src = String::Flatten(isolate, src);
9.    StringSharedKey key(src, shared, language_mode, kNoSourcePosition);
10.    Handle<Object> k = key.AsHandle(isolate);
11.    cache = EnsureCapacity(isolate, cache, 1);
12.    int entry = cache->FindInsertionEntry(key.Hash());
13.    cache->set(EntryToIndex(entry), *k);
14.    cache->set(EntryToIndex(entry) + 1, *value);
15.    cache->ElementAdded();
16.    return cache;
17.  } I think the important thing is that PutScript binds the JavaScript source and SharedFuncation as a hash pair as well as puts it into the cache, which the implementations are from line 9 to line 15. 2. Lookup and update Let’s go into Compiler::GetSharedFunctionInfoForScript. 1.   MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
2.       Isolate* isolate, Handle<String> source,
3.       const Compiler::ScriptDetails& script_details,
4.       ScriptOriginOptions origin_options, v8::Extension* extension,
5.       ScriptData* cached_data, ScriptCompiler::CompileOptions compile_options,
6.       ScriptCompiler::NoCacheReason no_cache_reason, NativesFlag natives) {
7.  //omit.....................
8.       if (extension == nullptr) {
9.         bool can_consume_code_cache =
10.             compile_options == ScriptCompiler::kConsumeCodeCache;
11.         if (can_consume_code_cache) {
12.           compile_timer.set_consuming_code_cache();
13.         }
14.         maybe_result = compilation_cache->LookupScript(//lookup is here.
15.             source, script_details.name_obj, script_details.line_offset,
16.             script_details.column_offset, origin_options, isolate->native_context(),
17.             language_mode);
18.         if (!maybe_result.is_null()) {
19.           compile_timer.set_hit_isolate_cache();
20.         } else if (can_consume_code_cache) {
21.  //omit
22.           if (CodeSerializer::Deserialize(isolate, cached_data, source,
23.                                           origin_options)
24.                   .ToHandle(&inner_result) &&
25.               inner_result->is_compiled()) {
26.             is_compiled_scope = inner_result->is_compiled_scope();
27.             DCHECK(is_compiled_scope.is_compiled());
28.     		//filling is here
29.             compilation_cache->PutScript(source, isolate->native_context(),
30.                                          language_mode, inner_result);
31.             Handle<Script> script(Script::cast(inner_result->script()), isolate);
32.             maybe_result = inner_result;
33.           } else {
34.             compile_timer.set_consuming_code_cache_failed();
35.           }
36.         }
37.       }
38.       return maybe_result;
39.     } In line 14, it means that we need to compile the JavaScript code if LookupScript is missing, so we skip to line 21 to perform compilation and generate inner_result. Then at line 29, put the inner_result into the cache. We can find that the innuer_result is just the maybe_result at line 32. Figure 1 shows the call stack. Besides LookupScript and PutScript, there are also LookupEval, PutEval as well as LookupRegExp, PutRegExp in V8. The workflow of them is the same as LookupScript and PutScript, please learn by yourself. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. qq9123013 v8blink@outlook.com WeChat: Email: Also published . here

Understanding Chrome V8: Chapter 19: Compilation Cache — Make the Compiler Faster

Execution is responsible for executing the bytecode. In this paper, we’ll talk about the workflow from S to E and several kernel functions. Our test case: function JsPrint(a){
	if(a >5){
		return "Debug";
	}
}
console.log(JsPrint(6)); Workflow The GetSharedFunctionInfoForScript() compiles JS source code into bytecodes. 0.  MaybeHandle<SharedFunctionInfo> Compiler::GetSharedFunctionInfoForScript(
1.      Isolate* isolate, Handle<String> source,
2.      const Compiler::ScriptDetails& script_details,
3.      ScriptOriginOptions origin_options, v8::Extension* extension,
4.      ScriptData* cached_data, ScriptCompiler::CompileOptions compile_options,
5.      ScriptCompiler::NoCacheReason no_cache_reason, NativesFlag natives) {
6.  //.omit.............
7.    // Do a lookup in the compilation cache but not for extensions.
8.    MaybeHandle<SharedFunctionInfo> maybe_result;
9.    IsCompiledScope is_compiled_scope;
10.    if (extension == nullptr) {
11.      bool can_consume_code_cache =
12.          compile_options == ScriptCompiler::kConsumeCodeCache;
13.      if (can_consume_code_cache) {
14.        compile_timer.set_consuming_code_cache();
15.      }
16.      // First check per-isolate compilation cache.
17.      maybe_result = compilation_cache->LookupScript(
18.          source, script_details.name_obj, script_details.line_offset,
19.          script_details.column_offset, origin_options, isolate->native_context(),
20.          language_mode);
21.      if (!maybe_result.is_null()) {
22.        compile_timer.set_hit_isolate_cache();
23.      } 
24.    }
25.  //.....omit..............
26.    if (maybe_result.is_null()) {
27.      ParseInfo parse_info(isolate);
28.      // No cache entry found compile the script.
29.      NewScript(isolate, &parse_info, source, script_details, origin_options,
30.                natives);
31.      // Compile the function and add it to the isolate cache.
32.      if (origin_options.IsModule()) parse_info.set_module();
33.      parse_info.set_extension(extension);
34.      parse_info.set_eager(compile_options == ScriptCompiler::kEagerCompile);
35.      parse_info.set_language_mode(
36.          stricter_language_mode(parse_info.language_mode(), language_mode));
37.      maybe_result = CompileToplevel(&parse_info, isolate, &is_compiled_scope);
38.      Handle<SharedFunctionInfo> result;
39.      if (extension == nullptr && maybe_result.ToHandle(&result)) {
40.        DCHECK(is_compiled_scope.is_compiled());
41.        compilation_cache->PutScript(source, isolate->native_context(),
42.                                     language_mode, result);
43.      } else if (maybe_result.is_null() && natives != EXTENSION_CODE) {
44.        isolate->ReportPendingMessages();
45.      }
46.    }
47.    return maybe_result;
48.  } In the 17th line of code, lookup compilation_cache, cache hit indicates that there are already bytecodes, and don’t need to recompile. The 26th line of code indicates that cache miss, so call the CompileToplevel on the 37th line to generate bytecodes. Bytecode needs to be bound to a context before execution, as below: 0.  Local<Script> UnboundScript::BindToCurrentContext() {
1.    auto function_info =
2.        i::Handle<i::SharedFunctionInfo>::cast(Utils::OpenHandle(this));
3.    i::Isolate* isolate = function_info->GetIsolate();
4.    i::Handle<i::JSFunction> function =
5.        isolate->factory()->NewFunctionFromSharedFunctionInfo(
6.            function_info, isolate->native_context());
7.    return ToApiHandle<Script>(function);
8.  } After binding, we get an important member — JSFunction, which is the input expected by the execution. 0.  bool Shell::ExecuteString(Isolate* isolate, Local<String> source,
1.                            Local<Value> name, PrintResult print_result,
2.                            ReportExceptions report_exceptions,
3.                            ProcessMessageQueue process_message_queue) {
4.    bool success = true;
5.    {
6.      PerIsolateData* data = PerIsolateData::Get(isolate);
7.      Local<Context> realm =
8.          Local<Context>::New(isolate, data->realms_[data->realm_current_]);
9.      Context::Scope context_scope(realm);
10.      MaybeLocal<Script> maybe_script;
11.      Local<Context> context(isolate->GetCurrentContext());
12.      ScriptOrigin origin(name);
13.      if (options.compile_options == ScriptCompiler::kConsumeCodeCache) {
14.  //.......omit
15.      } else if (options.stress_background_compile) {
16.  //.......omit
17.      } else {
18.        ScriptCompiler::Source script_source(source, origin);
19.        maybe_script = ScriptCompiler::Compile(context, &script_source,
20.                                               options.compile_options);
21.      }
22.      maybe_result = script->Run(realm);
23.  //......omit
24.      if (options.code_cache_options ==
25.          ShellOptions::CodeCacheOptions::kProduceCacheAfterExecute) {
26.        // Serialize and store it in memory for the next execution.
27.        ScriptCompiler::CachedData* cached_data =
28.            ScriptCompiler::CreateCodeCache(script->GetUnboundScript());
29.        StoreInCodeCache(isolate, source, cached_data);
30.        delete cached_data;
31.      }
32.      if (process_message_queue && !EmptyMessageQueues(isolate)) success = false;
33.      data->realm_current_ = data->realm_switch_;
34.    }
35.    DCHECK(!try_catch.HasCaught());
36.    return success;
37.  } The above function is the entrance for executing JS. Line 19 compiles the JS to bytecode. Line 22 executes the bytecode. Debug line 22 to step into the Run(realm). 0.  MaybeLocal<Value> Script::Run(Local<Context> context) {
1.    auto isolate = reinterpret_cast<i::Isolate*>(context->GetIsolate());
2.    TRACE_EVENT_CALL_STATS_SCOPED(isolate, "v8", "V8.Execute");
3.    ENTER_V8(isolate, context, Script, Run, MaybeLocal<Value>(),
4.             InternalEscapableScope);
5.    i::HistogramTimerScope execute_timer(isolate->counters()->execute(), true);
6.    i::AggregatingHistogramTimerScope timer(isolate->counters()->compile_lazy());
7.    i::TimerEventScope<i::TimerEventExecute> timer_scope(isolate);
8.    auto fun = i::Handle<i::JSFunction>::cast(Utils::OpenHandle(this));
9.    i::Handle<i::Object> receiver = isolate->global_proxy();
10.    Local<Value> result;
11.    has_pending_exception = !ToLocal<Value>(
12.        i::Execution::Call(isolate, fun, receiver, 0, nullptr), &result);
13.    RETURN_ON_FAILED_EXECUTION(Value);
14.    RETURN_ESCAPED(result);
15.  } The 8th line casts this pointer to JSFunction. For our case, the variable fun is console.log(JsPrint(6)), then call the 12th line — call(). In call(), the Invoke() will be called which is below. 0.  V8_WARN_UNUSED_RESULT MaybeHandle<Object> Invoke(Isolate* isolate,
1.                                                   const InvokeParams& params) {
2.  //...............omit
3.    Object value;
4.    Handle<Code> code =
5.        JSEntry(isolate, params.execution_target, params.is_construct);
6.    {
7.      SaveContext save(isolate);
8.      SealHandleScope shs(isolate);
9.      if (FLAG_clear_exceptions_on_js_entry) isolate->clear_pending_exception();
10.      if (params.execution_target == Execution::Target::kCallable) {
11.        // clang-format off
12.        // {new_target}, {target}, {receiver}, return value: tagged pointers
13.        // {argv}: pointer to array of tagged pointers
14.        using JSEntryFunction = GeneratedCode<Address(
15.            Address root_register_value, Address new_target, Address target,
16.            Address receiver, intptr_t argc, Address** argv)>;
17.        // clang-format on
18.        JSEntryFunction stub_entry =
19.            JSEntryFunction::FromAddress(isolate, code->InstructionStart());
20.        Address orig_func = params.new_target->ptr();
21.        Address func = params.target->ptr();
22.        Address recv = params.receiver->ptr();
23.        Address** argv = reinterpret_cast<Address**>(params.argv);
24.        RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::kJS_Execution);
25.        value = Object(stub_entry.Call(isolate->isolate_data()->isolate_root(),
26.                                       orig_func, func, recv, params.argc, argv));
27.      } else {
28.  //..................omit     
29.      }
30.    } Line 18 gets the address of the builtin, its index is 40, and its name is JSEntry. This builtin builds the execution environment for bytecodes. Then step into line 25, where the bytecode is executed. After that, we can only debug the assembly code, but c++. Figure 1 is the call stack. Before stepping into bytecodes, Generate_JSEntryVariant will be called. 0.  void Generate_JSEntryVariant(MacroAssembler* masm, StackFrame::Type type,
1.                               Builtins::Name entry_trampoline) {
2.    Label invoke, handler_entry, exit;
3.    Label not_outermost_js, not_outermost_js_2;
4.    {  // NOLINT. Scope block confuses linter.
5.      NoRootArrayScope uninitialized_root_register(masm);
6.      // Set up frame.
7.      __ pushq(rbp);
8.      __ movq(rbp, rsp);
9.      // Push the stack frame type.
10.      __ Push(Immediate(StackFrame::TypeToMarker(type)));
11.      // Reserve a slot for the context. It is filled after the root register has
12.      // been set up.
13.      __ AllocateStackSpace(kSystemPointerSize);
14.      // Save callee-saved registers (X64/X32/Win64 calling conventions).
15.      __ pushq(r12);
16.      __ pushq(r13);
17.      __ pushq(r14);
18.      __ pushq(r15);
19.  #ifdef _WIN64
20.      __ pushq(rdi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
21.      __ pushq(rsi);  // Only callee save in Win64 ABI, argument in AMD64 ABI.
22.  #endif
23.      __ pushq(rbx);
24.  //.....................omit............................
25.    __ ret(0);
26.  }
27.  }  // namespace
28.  //==================separation===============================
29.  void Builtins::Generate_JSEntry(MacroAssembler* masm) {
30.    Generate_JSEntryVariant(masm, StackFrame::ENTRY,
31.                            Builtins::kJSEntryTrampoline);
32.  } Look at the 30th line of code, the generator function of the 40th builtin is B, and its analysis method refers to my previous paper. Figure 2 shows the corresponding assembly code. You can also see a bit of C++ in Figure 2, but this is the last chance to see C++. After that, only the assembly code. Our case uses the console.log which is a global function. It is below. 1.  #define RUNTIME_FUNCTION_RETURNS_TYPE(Type, InternalType, Convert, Name)      \
2.    static V8_INLINE InternalType __RT_impl_##Name(Arguments args,              \
3.                                                   Isolate* isolate);           \
4.                                                                                \
5.    V8_NOINLINE static Type Stats_##Name(int args_length, Address* args_object, \
6.                                         Isolate* isolate) {                    \
7.      RuntimeCallTimerScope timer(isolate, RuntimeCallCounterId::k##Name);      \
8.      TRACE_EVENT0(TRACE_DISABLED_BY_DEFAULT("v8.runtime"),                     \
9.                   "V8.Runtime_" #Name);                                        \
10.     Arguments args(args_length, args_object);                                 \
11.      return Convert(__RT_impl_##Name(args, isolate));                          \
12.    }                                                                           \
13.                                                                                \
14.    Type Name(int args_length, Address* args_object, Isolate* isolate) {        \
15.      DCHECK(isolate->context().is_null() || isolate->context().IsContext());   \
16.      CLOBBER_DOUBLE_REGISTERS();                                               \
17.      if (V8_UNLIKELY(TracingFlags::is_runtime_stats_enabled())) {              \
18.        return Stats_##Name(args_length, args_object, isolate);                 \
19.      }                                                                         \
20.      Arguments args(args_length, args_object);                                 \
21.      return Convert(__RT_impl_##Name(args, isolate));                          \
22.    }                                                                           \
23.                                                                                \
24.    static InternalType __RT_impl_##Name(Arguments args, Isolate* isolate)
25.  #define CONVERT_OBJECT(x) (x).ptr()
26.  #define CONVERT_OBJECTPAIR(x) (x)
27.  #define RUNTIME_FUNCTION(Name) \
28.    RUNTIME_FUNCTION_RETURNS_TYPE(Address, Object, CONVERT_OBJECT, Name)
29.  //=================separation==============================
30.  //==================separation===============================
31.  Object DeclareGlobals(Isolate* isolate, Handle<FixedArray> declarations,
32.                        int flags, Handle<JSFunction> closure) {
33.    HandleScope scope(isolate);
34.    Handle<JSGlobalObject> global(isolate->global_object());
35.    Handle<Context> context(isolate->context(), isolate);
36.    Handle<FeedbackVector> feedback_vector = Handle<FeedbackVector>::null();
37.    Handle<ClosureFeedbackCellArray> closure_feedback_cell_array =
38.        Handle<ClosureFeedbackCellArray>::null();
39.    if (closure->has_feedback_vector()) {
40.      feedback_vector =
41.          Handle<FeedbackVector>(closure->feedback_vector(), isolate);
42.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
43.          feedback_vector->closure_feedback_cell_array(), isolate);
44.    } else {
45.      closure_feedback_cell_array = Handle<ClosureFeedbackCellArray>(
46.          closure->closure_feedback_cell_array(), isolate);
47.    }
48.    // Traverse the name/value pairs and set the properties.
49.    int length = declarations->length();
50.    FOR_WITH_HANDLE_SCOPE(isolate, int, i = 0, i, i < length, i += 4, {
51.      Handle<String> name(String::cast(declarations->get(i)), isolate);
52.      FeedbackSlot slot(Smi::ToInt(declarations->get(i + 1)));
53.      Handle<Object> possibly_feedback_cell_slot(declarations->get(i + 2),
54.                                                 isolate);
55.      Handle<Object> initial_value(declarations->get(i + 3), isolate);
56.      bool is_var = initial_value->IsUndefined(isolate);
57.      bool is_function = initial_value->IsSharedFunctionInfo();
58.      DCHECK_NE(is_var, is_function);
59.      Handle<Object> value;
60.      if (is_function) {
61.        DCHECK(possibly_feedback_cell_slot->IsSmi());
62.        Handle<FeedbackCell> feedback_cell =
63.            closure_feedback_cell_array->GetFeedbackCell(
64.                Smi::ToInt(*possibly_feedback_cell_slot));
65.        // Copy the function and update its context. Use it as value.
66.        Handle<SharedFunctionInfo> shared =
67.            Handle<SharedFunctionInfo>::cast(initial_value);
68.        Handle<JSFunction> function =
69.            isolate->factory()->NewFunctionFromSharedFunctionInfo(
70.                shared, context, feedback_cell, AllocationType::kOld);
71.        value = function;
72.      } else {
73.        value = isolate->factory()->undefined_value();
74.      }
75.      // Compute the property attributes. According to ECMA-262,
76.      // the property must be non-configurable except in eval.
77.      bool is_eval = DeclareGlobalsEvalFlag::decode(flags);
78.      int attr = NONE;
79.      if (!is_eval) attr |= DONT_DELETE;
80.      // ES#sec-globaldeclarationinstantiation 5.d:
81.      // If hasRestrictedGlobal is true, throw a SyntaxError exception.
82.      Object result = DeclareGlobal(isolate, global, name, value,
83.                                    static_cast<PropertyAttributes>(attr), is_var,
84.                                    is_function, RedeclarationType::kSyntaxError,
85.                                    feedback_vector, slot);
86.      if (isolate->has_pending_exception()) return result;
87.    });
88.    return ReadOnlyRoots(isolate).undefined_value();
89.  }
90.  //==================separation===============================
91.  //==================separation===============================
92.  RUNTIME_FUNCTION(Runtime_DeclareGlobals) {
93.    HandleScope scope(isolate);
94.    DCHECK_EQ(3, args.length());
95.    CONVERT_ARG_HANDLE_CHECKED(FixedArray, declarations, 0);
96.    CONVERT_SMI_ARG_CHECKED(flags, 1);
97.    CONVERT_ARG_HANDLE_CHECKED(JSFunction, closure, 2);
98.    return DeclareGlobals(isolate, declarations, flags, closure);
99.  } RUNTIME_FUNCTION(Runtime_DeclareGlobals) is a runtime function defined by a macro template. The function checks the parameters and then calls DeclareGlobals() to perform the corresponding operation. In our case, DeclareGlobals() lookup the global object console, then takes out the log() from the console, and gets the address of console.log(). Finally, pass the parameter JsPrint(6) into log() which will start a new compilation and execution. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : v8blink@outlook.com WeChat Email This story was first published here.

SCRIPTO

scripto.live

-- employees

Since n.d.

COMPANY RANKING

SCRIPTO

EVERGREEN INDEX #9605

Representing this company? Click here to claim and customize this page!

Scripto WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

SCRIPTO

scripto.live

-- employees

Since n.d.

COMPANY RANKING

SCRIPTO

EVERGREEN INDEX #9605

RELATED COMPANIESRANK

STORIES ABOUT SCRIPTO

Representing this company? Click here to claim and customize this page!

Scripto WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!