OStream is a data infrastructure company enabling scalable Edge AI media processing. Physico, an edge platform to objectify media streams for actions at the edge and intuitive media search at scale. Integrators use Physico’s Edge AI to enrich media streams with object data for real-time event generation and automatic analysis of physical spaces. Physical risks are just as urgent and consequential as digital ones… they affect customers, employees, product quality and critical assets. Physico does for media what IT has already handled with machine data. Edge Platform to Objectify Media Streams: Actions at the Edge and Intuitive Search at Scale.

OStream

ostream

Welcome to other chapters of Let’s Understand Chrome V8 Dispatch is responsible for scheduling bytecode, which is equivalent to registering EIP++, and jumps to the next bytecode. Dispatch consists of two parts, one is a dispatch table and the other is the physical register. The table is an array that contains all bytecode addresses. V8 uses the physical register to dispatch bytecode for getting greater efficiency. 1. Dispatch Table The dispatch table is an array of pointers, which is used to store the address of the bytecode processing program. The array type is Code, and the Code Class is below: class Code: public HeapObject {
 public: NEVER_READ_ONLY_SPACE
 using Flags = uint32_t;
 #define CODE_KIND_LIST(V)\
 V(OPTIMIZED_FUNCTION)\
 V(BYTECODE_HANDLER)\
 V(STUB)\
 V(BUILTIN)\
 V(REGEXP)\
 V(WASM_FUNCTION)\
 V(WASM_TO_CAPI_FUNCTION)\
 V(WASM_TO_JS_FUNCTION)\
 V(JS_TO_WASM_FUNCTION)\
 V(JS_TO_JS_FUNCTION)\
 V(WASM_INTERPRETER_ENTRY)\
 V(C_WASM_ENTRY)
 enum Kind {
 #define DEFINE_CODE_KIND_ENUM(name) name,
 CODE_KIND_LIST(DEFINE_CODE_KIND_ENUM)
 #undef DEFINE_CODE_KIND_ENUM
 NUMBER_OF_KINDS
 };
 static
 const char * Kind2String(Kind kind);
 #ifdef ENABLE_DISASSEMBLER
 const char * GetName(Isolate * isolate) const;
 V8_EXPORT_PRIVATE void Disassemble(const char * name, std::ostream & os,
 Address current_pc = kNullAddress);
 #endif
 //................omit................
 }; : In the isolate, there is another pointer array that is the same type as the dispatch table. It holds all the Builtins addresses, which can confuse beginners easily. Note The dispatch table is maintained by BuildWithMacroAssembler, code is below: Code BuildWithMacroAssembler(Isolate * isolate, int32_t builtin_index,
 MacroAssemblerGenerator generator,
 const char * s_name) {
 HandleScope scope(isolate);
 // Canonicalize handles, so that we can share constant pool entries pointing
 // to code targets without dereferencing their handles.
 CanonicalHandleScope canonical(isolate);
 constexpr int kBufferSize = 32 * KB;
 byte buffer[kBufferSize];
 MacroAssembler masm(isolate, BuiltinAssemblerOptions(isolate, builtin_index),
 CodeObjectRequired::kYes,
 ExternalAssemblerBuffer(buffer, kBufferSize));
 masm.set_builtin_index(builtin_index);
 DCHECK(!masm.has_frame());
 generator( & masm);
 int handler_table_offset = 0;
 // JSEntry builtins are a special case and need to generate a handler table.
 DCHECK_EQ(Builtins::KindOf(Builtins::kJSEntry), Builtins::ASM);
 DCHECK_EQ(Builtins::KindOf(Builtins::kJSConstructEntry), Builtins::ASM);
 DCHECK_EQ(Builtins::KindOf(Builtins::kJSRunMicrotasksEntry), Builtins::ASM);
 if (Builtins::IsJSEntryVariant(builtin_index)) {
 handler_table_offset = HandlerTable::EmitReturnTableStart( & masm);
 HandlerTable::EmitReturnEntry( &
 masm, 0, isolate -> builtins() -> js_entry_handler_offset());
 }
 //.....................................................
 //................omit.............................
 } You can see regarding how to debug the BuildWithMacroAssembler. When the value of the parameter builtin_index is 65, the function pointer generator points to which is responsible for maintaining the Dispatch table, code is below: Chapter 9 Generate_InterpreterEnterBytecodeDispatch static void Generate_InterpreterEnterBytecode(MacroAssembler * masm) {
 // Set the return address to the correct point in the interpreter entry
 // trampoline.
 Label builtin_trampoline, trampoline_loaded;
 Smi interpreter_entry_return_pc_offset(
 masm -> isolate() -> heap() -> interpreter_entry_return_pc_offset());
 DCHECK_NE(interpreter_entry_return_pc_offset, Smi::kZero);
 // If the SFI function_data is an InterpreterData, the function will have a
 // custom copy of the interpreter entry trampoline for profiling. If so,
 // get the custom trampoline, otherwise grab the entry address of the global
 // trampoline.
 __ movq(rbx, Operand(rbp, StandardFrameConstants::kFunctionOffset));
 __ LoadTaggedPointerField(
 rbx, FieldOperand(rbx, JSFunction::kSharedFunctionInfoOffset));
 __ LoadTaggedPointerField(
 rbx, FieldOperand(rbx, SharedFunctionInfo::kFunctionDataOffset));
 __ CmpObjectType(rbx, INTERPRETER_DATA_TYPE, kScratchRegister);
 __ j(not_equal, & builtin_trampoline, Label::kNear);
 __ movq(rbx,
 FieldOperand(rbx, InterpreterData::kInterpreterTrampolineOffset));
 __ addq(rbx, Immediate(Code::kHeaderSize - kHeapObjectTag));
 __ jmp( & trampoline_loaded, Label::kNear);
 __ bind( & builtin_trampoline);
 // TODO(jgruber): Replace this by a lookup in the builtin entry table.
 __ movq(rbx,
 __ ExternalReferenceAsOperand(
 ExternalReference::
 address_of_interpreter_entry_trampoline_instruction_start(
 masm -> isolate()),
 kScratchRegister));
 __ bind( & trampoline_loaded);
 __ addq(rbx, Immediate(interpreter_entry_return_pc_offset.value()));
 __ Push(rbx);
 // Initialize dispatch table register.
 __ Move(
 kInterpreterDispatchTableRegister,
 ExternalReference::interpreter_dispatch_table_address(masm -> isolate()));
 // Get the bytecode array pointer from the frame.
 __ movq(kInterpreterBytecodeArrayRegister,
 Operand(rbp, InterpreterFrameConstants::kBytecodeArrayFromFp));
 if (FLAG_debug_code) {
 // Check function data field is actually a BytecodeArray object.
 __ AssertNotSmi(kInterpreterBytecodeArrayRegister);
 __ CmpObjectType(kInterpreterBytecodeArrayRegister, BYTECODE_ARRAY_TYPE,
 rbx);
 __ Assert(
 equal,
 AbortReason::kFunctionDataShouldBeBytecodeArrayOnInterpreterEntry);
 }
 // Get the target bytecode offset from the frame.
 __ movq(kInterpreterBytecodeOffsetRegister,
 Operand(rbp, InterpreterFrameConstants::kBytecodeOffsetFromFp));
 __ SmiUntag(kInterpreterBytecodeOffsetRegister,
 kInterpreterBytecodeOffsetRegister);
 // Dispatch to the target bytecode.
 __ movzxbq(r11, Operand(kInterpreterBytecodeArrayRegister,
 kInterpreterBytecodeOffsetRegister, times_1, 0));
 __ movq(kJavaScriptCallCodeStartRegister,
 Operand(kInterpreterDispatchTableRegister, r11,
 times_system_pointer_size, 0));
 __ jmp(kJavaScriptCallCodeStartRegister);
 }
 //================================separation=========================
 void Builtins::Generate_InterpreterEnterBytecodeDispatch(MacroAssembler * masm) {
 Generate_InterpreterEnterBytecode(masm);
 } In the above code, the is the entry point of bytecode dispatch, and the is the key code that implements the bytecode dispatch. Here are two important concepts: Generate_InterpreterEnterBytecodeDispatch Generate_InterpreterEnterBytecode The is a Builtin and its index is 65. (The index may be dissimilar in different V8 versions) Generate_InterpreterEnterBytecodeDispatch V8 uses a physical register to manage the dispatch table, the name of the register is . Using the physical register can avoid the push and pop of the dispatch table frequently, to improve efficiency. kInterpreterDispatchTableRegister Here are the key lines of code： Line 35, loads the dispatch table to the A register. Line 37 of code, take out the table address from the isolate. The implementation of A is in the below three functions. (1) ExternalReference ExternalReference::interpreter_dispatch_table_address(
 Isolate * isolate) {
 return ExternalReference(isolate -> interpreter() -> dispatch_table_address());
}
interpreter::Interpreter * interpreter() const {
 return interpreter_;
}
Address dispatch_table_address() {
 return reinterpret_cast < Address > ( & dispatch_table_[0]);
} Line 39 takes out the bytecode array from the stack. (2) Line 51 takes out the offset of the target bytecode from the bytecode array. (3) Lines 56 and 58 calculate the target bytecode address and store the address in kJavaScriptCallCodeStartRegister. (4) Line 61 jumps to register kJavaScriptCallCodeStartRegister and executes bytecode. Figure 1 is the call stack. (5) 2. TailCall The Dispatch() is the last step in a bytecode, so its alias is the tail call. Let’s look at a few bytecodes： IGNITION_HANDLER(StaGlobal, InterpreterAssembler) {
 TNode < Context > context = GetContext();
 //omit..............
 Goto( & end);
 Bind( & no_feedback);
 CallRuntime(Runtime::kStoreGlobalICNoFeedback_Miss, context, value, name);
 Goto( & end);
 Bind( & end);
 Dispatch(); // !!! this is the tail call.
 }
 IGNITION_HANDLER(LdaContextSlot, InterpreterAssembler) {
 TNode < Context > context = CAST(LoadRegisterAtOperandIndex(0));
 TNode < IntPtrT > slot_index = Signed(BytecodeOperandIdx(1));
 TNode < Uint32T > depth = BytecodeOperandUImm(2);
 TNode < Context > slot_context = GetContextAtDepth(context, depth);
 TNode < Object > result = LoadContextElement(slot_context, slot_index);
 SetAccumulator(result);
 Dispatch(); //!!! here also is.
 }
 IGNITION_HANDLER(LdaImmutableContextSlot, InterpreterAssembler) {
 TNode < Context > context = CAST(LoadRegisterAtOperandIndex(0));
 TNode < IntPtrT > slot_index = Signed(BytecodeOperandIdx(1));
 TNode < Uint32T > depth = BytecodeOperandUImm(2);
 TNode < Context > slot_context = GetContextAtDepth(context, depth);
 TNode < Object > result = LoadContextElement(slot_context, slot_index);
 SetAccumulator(result);
 Dispatch(); //!!! here 
 } The above three bytecodes all end with Dispatch(). Below is the code of Dispatch: void InterpreterAssembler::Dispatch() {
 Comment("========= Dispatch");
 DCHECK_IMPLIES(Bytecodes::MakesCallAlongCriticalPath(bytecode_), made_call_);
 TNode < IntPtrT > target_offset = Advance();
 TNode < WordT > target_bytecode = LoadBytecode(target_offset);
 if (Bytecodes::IsStarLookahead(bytecode_, operand_scale_)) {
 target_bytecode = StarDispatchLookahead(target_bytecode);
 }
 DispatchToBytecode(target_bytecode, BytecodeOffset());
 } The 2nd line is the comment function. When debugging, we can use its output information to locate the execution position. The 4th line takes out the offset of the target(next) bytecode. The 5th line loads the target bytecode. The 9th line calls which calls eventually. DspatchToBytecode() DispatchToBytecodeHandlerEntry() void InterpreterAssembler::DispatchToBytecodeHandlerEntry(
 TNode < RawPtrT > handler_entry, TNode < IntPtrT > bytecode_offset) {
 // Propagate speculation poisoning.
 TNode < RawPtrT > poisoned_handler_entry =
 UncheckedCast < RawPtrT > (WordPoisonOnSpeculation(handler_entry));
 TailCallBytecodeDispatch(InterpreterDispatchDescriptor {},
 poisoned_handler_entry, GetAccumulatorUnchecked(),
 bytecode_offset, BytecodeArrayTaggedPointer(),
 DispatchTablePointer());
} In , parameter 1 is the target bytecode and 2 is the offset of the bytecode. Parameter 1 is used to load the bytecode handler and 2 is used to load operands. Then call , which is below: DispatchToBytecodeHandlerEntry TailCallBytecodeDispatch template < class...TArgs >
 void CodeAssembler::TailCallBytecodeDispatch(
 const CallInterfaceDescriptor & descriptor, TNode < RawPtrT > target,
 TArgs...args) {
 DCHECK_EQ(descriptor.GetParameterCount(), sizeof...(args));
 auto call_descriptor = Linkage::GetBytecodeDispatchCallDescriptor(
 zone(), descriptor, descriptor.GetStackParameterCount());
 Node * nodes[] = {
 target,
 args...
 };
 CHECK_EQ(descriptor.GetParameterCount() + 1, arraysize(nodes));
 raw_assembler() -> TailCallN(call_descriptor, arraysize(nodes), nodes);
 } In the above code, the 5th line creates the descriptor, as below: CallDescriptor * Linkage::GetBytecodeDispatchCallDescriptor(
 Zone * zone,
 const CallInterfaceDescriptor & descriptor,
 int stack_parameter_count) {
 const int register_parameter_count = descriptor.GetRegisterParameterCount();
 const int parameter_count = register_parameter_count + stack_parameter_count;
 DCHECK_EQ(descriptor.GetReturnCount(), 1);
 LocationSignature::Builder locations(zone, 1, parameter_count);
 locations.AddReturn(regloc(kReturnRegister0, descriptor.GetReturnType(0)));
 for (int i = 0; i < parameter_count; i++) {
 if (i < register_parameter_count) {
 // The first parameters go in registers.
 Register reg = descriptor.GetRegisterParameter(i);
 MachineType type = descriptor.GetParameterType(i);
 locations.AddParam(regloc(reg, type));
 } else {
 int stack_slot = i - register_parameter_count - stack_parameter_count;
 locations.AddParam(LinkageLocation::ForCallerFrameSlot(
 stack_slot, MachineType::AnyTagged()));
 }
 }
 MachineType target_type = MachineType::Pointer();
 LinkageLocation target_loc = LinkageLocation::ForAnyRegister(target_type);
 const CallDescriptor::Flags kFlags =
 CallDescriptor::kCanUseRoots | CallDescriptor::kFixedTargetRegister;
 return new(zone) CallDescriptor( // --
 CallDescriptor::kCallAddress, // kind
 target_type, // target MachineType
 target_loc, // target location
 locations.Build(), // location_sig
 stack_parameter_count, // stack_parameter_count
 Operator::kNoProperties, // properties
 kNoCalleeSaved, // callee-saved registers
 kNoCalleeSaved, // callee-saved fp
 kFlags, // flags
 descriptor.DebugName());
 } In , parameter 2 is the target bytecode, and parameter 3 is the stack parameter count. This function is used to apply for register resources and create a . The gives what parameters to supply when executing a bytecode. The comments on lines 24 to 25 give its layout. GetBytecodeDispatchCallDescriptor CallDescriptor CallDescriptor In , the on the 9th line is below, we see that it’s parameter 1 is . TailCallBytecodeDispatch TailCallN() CallDescriptor void RawMachineAssembler::TailCallN(CallDescriptor * call_descriptor,
 int input_count, Node *
 const * inputs) {
 // +1 is for target.
 DCHECK_EQ(input_count, call_descriptor -> ParameterCount() + 1);
 Node * tail_call =
 MakeNode(common() -> TailCall(call_descriptor), input_count, inputs);
 schedule() -> AddTailCall(CurrentBlock(), tail_call);
 current_block_ = nullptr;
 } The 5th line generates a node. In the 7th line adds the node to the end of the current basic block. So far, the dispatch is completed and the next bytecode is started to execute. Figure 2 shows the function call stack. AddTailCall() Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : v8blink@outlook.com WeChat Email Also published . here

Let's Understand Chrome V8 — Chapter 11: Bytecode Dispatch 

Prototype Design Pattern is a Creational Design Pattern that . You can consider the prototype as a of an object before the actual object is constructed. In this article of the Creational Design Patterns, we’re going to take a look at why we need a Prototype Design Pattern in C++ i.e. motivation, prototype factory & leveraging prototype design pattern to implement . helps in the prototyping(creating/copying cheaply) of an object using separate methods or polymorphic classes template virtual copy constructor By the way, If you haven’t check out my other articles on Creational Design Patterns, then here is the list: Factory Builder Prototype Singleton The code snippets you see throughout this series of articles are simplified not sophisticated. So you often see me not using keywords like override, final, public(while inheritance) just to make code compact & consumable(most of the time) in single standard screen size. I also prefer struct instead of class just to save line by not writing “public:” sometimes and also miss , constructor, , prefix std::, deleting dynamic memory, intentionally. I also consider myself a pragmatic person who wants to convey an idea in the simplest way possible rather than the standard way or using Jargons. virtual destructor copy constructor Note: If you stumbled here directly, then I would suggest you go through first, even if it is trivial. I believe it will encourage you to explore more on this topic. What is design pattern? All of this code you encounter in this series of articles are compiled using C++20(though I have used features up to C++17 in most cases). So if you don’t have access to the latest compiler you can use which has preinstalled boost library as well. Modern C++ https://wandbox.org/ Intent To create a new object cheaply with the help of an already constructed or pre-initialized stored object. The prototype provides flexibility to create complex object cheaply. The concept is to copy an existing object rather than creating a new instance from scratch, something that may include costly operations. The existing object then acts as a prototype & newly copied object may change the same properties only if required. This approach saves costly resources and time, especially when the object creation is a heavy process. So essentially the prototype is quite simply a partially or fully initialized object that you make a copy of. And then you subsequently use for your own benefit with variations. Motivation m_street; m_city; m_cubical;

 Office( s, c, n):m_street(s), m_city(c), m_cubical(n){}
}; m_name;
 Office m_office;

 Employee( n, Office o):m_name(n), m_office(o){}
}; {
 Employee john{ , Office{ , , } };
 Employee jane{ , Office{ , , } };
 Employee jack{ , Office{ , , } }; EXIT_SUCCESS;
} { struct Office string string int32_t string string int32_t { struct Employee string string int main () "John Doe" "123 East Dr" "London" 123 "Jane Doe" "123 East Dr" "London" 124 "jack Doe" "123 ORR" "Bangaluru" 300 return This is not the right approach as you have to write the main office address again & again for each employee detail. This is cumbersome & become more when you want to create an employee list. Moreover, consider the situation when your main office moved to another address. Prototype Design Pattern Examples in C++ A more pragmatic approach would be like this : m_name; Office* m_office; 
 
 Employee( n, Office *o):m_name(n), m_office(o){}
}; Office LondonOffice{ , , }; Office BangaluruOffice{ , , }; {
 Employee john{ , &LondonOffice };
 Employee jane{ , &LondonOffice };
 Employee jack{ , &BangaluruOffice }; EXIT_SUCCESS;
} { struct Employee string const string static "123 East Dr" "London" 123 static "RMZ Ecoworld ORR" "London" 123 int main () "John Doe" "Jane Doe" "jack Doe" return Above solution is suitable for our use case but sometimes we want to customize that office address. And when it comes to pointers & references and any sort of indirection, ordinary copying using operator equals quite simply does not work. A standard way to implement this is by implementing the copy constructor Prototype Factory So in the previous example of the Prototype Design Pattern, we basically had a global for office addresses and used their address for creating prototypes. object Now, this isn’t particularly convenient to the consumers of your API because you might want to give them a prototype to work with. And you should explicit enough in terms of letting people know there is the only a unified way by which they create instances from a prototype and so that they cannot make individual instances by themselves. And in this case, what you would build is off-course is a Prototype Factory: m_street; m_city; m_cubical;
}; m_name;
 Office* m_office; Employee( n, Office *o) : m_name(n), m_office(o) {} :
 Employee( Employee &rhs) : m_name{rhs.m_name}, m_office{ Office{*rhs.m_office}} 
 { }

 Employee& =( Employee &rhs) { ( == &rhs) * ;
 m_name = rhs.m_name;
 m_office = Office{*rhs.m_office}; * ;
 } ostream & <<(ostream &os, Employee &o) { os << o.m_name << << o.m_office->m_street << << o.m_office->m_city << << o.m_office->m_cubical;
 }
}; Employee main; Employee aux; <Employee> NewEmployee( n, c, Employee &proto) { e = make_unique<Employee>(proto);
 e->m_name = n;
 e->m_office->m_cubical = c; e;
 } : <Employee> NewMainOfficeEmployee( name, cubical) { NewEmployee(name, cubical, main);
 } <Employee> NewAuxOfficeEmployee( name, cubical) { NewEmployee(name, cubical, aux);
 }
}; Employee EmployeeFactory::main{ , Office{ , , }};
Employee EmployeeFactory::aux{ , Office{ , , }}; { jane = EmployeeFactory::NewMainOfficeEmployee( , ); jack = EmployeeFactory::NewAuxOfficeEmployee( , ); << *jane << << *jack << ; EXIT_SUCCESS;
} { struct Office string string int32_t { class Employee string // Private constructor, so direct instance can not be created except for `class EmployeeFactory` string friend ; class EmployeeFactory public const new operator const if this return this new return this friend operator const return " works at " " " " seats @" { class EmployeeFactory static static static unique_ptr string int32_t auto return public static unique_ptr string int32_t return static unique_ptr string int32_t return // Static Member Initialization "" new "123 East Dr" "London" 123 "" new "RMZ Ecoworld ORR" "London" 123 int main () auto "Jane Doe" 125 auto "jack Doe" 123 cout endl endl return /*
Jane Doe works at 123 East Dr London seats @125
jack Doe works at RMZ Ecoworld ORR London seats @123
*/ The subtle thing to note here is the private constructor of Employee & friend EmployeeFactory. This is how we enforce the client/API-user to create an instance of Employee only through EmployeeFactory. Leveraging Prototype Design Pattern to Implement Virtual Copy Constructor In C++, Prototype is also useful to create a copy of an object without knowing its concrete type. Hence, it is also known as Virtual Copy Constructor. Problem C++ has the support of polymorphic object destruction using it’s base class’s . Equivalent support for creation and copying of objects is missing as С++ doesn’t support virtual constructor & virtual . virtual destructor copy constructors Moreover, you can’t create an object unless you know its static type, because the compiler must know the amount of space it needs to allocate. For the same reason, copy of an object also requires its type to known at compile-time. Consider the following example as problem statement: ~animal(){ << ; }
}; animal {
 ~dog(){ << ; }
}; animal {
 ~cat(){ << ; }
}; { who; } { struct animal virtual cout "~animal\n" : struct dog cout "~dog\n" : struct cat cout "~cat\n" void who_am_i (animal *who) // not sure whether dog would be passed here or cat // How to `create` the object of same type i.e. pointed by who ? // How to `copy` object of same type i.e. pointed by who ? delete // you can delete appropriate object pointed by who, thanks to virtual destructor Just don’t think of , its code smell. dynamic_cast<> Solution The Virtual Constructor/Copy-Constructor technique allows polymorphic creation & copying of objects in C++ by delegating the act of creation & copying the object to the derived class through the use of virtual methods. Following code is not only implements virtual (i.e. clone()) but also implement virtual constructor(i.e. create()). copy constructor ~animal() = ; :: <animal> create() = ; :: <animal> clone() = ;
}; animal { :: <animal> create() { ::make_unique<dog>(); } :: <animal> clone() { ::make_unique<dog>(* ); }
}; animal { :: <animal> create() { ::make_unique<cat>(); } :: <animal> clone() { ::make_unique<cat>(* ); }
}; { new_who = who->create(); duplicate_who = who->clone(); who; 
} { struct animal virtual default virtual std unique_ptr 0 virtual std unique_ptr 0 : struct dog std unique_ptr return std std unique_ptr return std this : struct cat std unique_ptr return std std unique_ptr return std this void who_am_i (animal *who) auto // `create` the object of same type i.e. pointed by who ? auto // `copy` object of same type i.e. pointed by who ? delete Benefits of Prototype Design Pattern Prototypes are useful when the object instantiation is expensive, thus avoid expensive “creation from scratch”, and support cheap cloning of a pre-initialized prototype. The prototype provides the flexibility to create highly dynamic systems by defining new behaviour through object & specifying values for an object’s data members at the time of instantiation unlike defining new classes. composition You can simplify the system by producing complex objects more conveniently. Especially in C++, Prototype Design Pattern is helpful in creating copy of an object without even knowing its type. Summary by FAQs What’s the point of using the Prototype Design Pattern? To create an object rapidly based on cloning a pre-configured object. Useful to remove a bunch of boilerplate code. Handly while working with object without knowing its type. Prototype Design Pattern is an obvious choice while you are working with the . For example, in HTTP request most of the time header & footer content remains the same, what changes are data. In such a scenario, you should not create an object from scratch. Rather leverage Prototype Design Pattern. Command Design Pattern Is the Prototype Design Pattern Really Just Clone? It isn’t if you combine it with the . Factory Design Pattern Prototype design pattern to be used when creation is costly, but we do create in the clone. You must be wondering that in Prototype Factory we show above, we are creating instances in the copy constructor. Isn’t that expensive. Yes, it is. But just think about HTTP request, its header consist version, encoding type, content type, server-type, etc. Initially, you need a find out these parameters using respective function calls. But once you got these, these are not going to change until connection closed. So there is no point in doing function calls to extract these params over & over. What cost us here is not parameters but their functions to extract value. Previously published at http://www.vishalchovatiya.com/prototype-design-pattern-in-modern-cpp/

Design Patterns: Prototype Pattern in Modern C++

In software engineering, Creational Design Patterns deal with object creation mechanisms, trying to create objects in a manner suitable to the situation. The basic or ordinary form of object creation could result in design problems or added complexity to the design. Builder Design Pattern in C++ solves this specific problem by . separating the construction of a complex object from its representation By the way, If you haven’t check out my other articles on Creational Design Patterns, then here is the list: Factory Builder Prototype Singleton The code snippets you see throughout this series of articles are simplified not sophisticated. So you often see me not using keywords like override, final, public(while inheritance) just to make code compact & consumable(most of the time) in single standard screen size. I also prefer struct instead of class just to save line by not writing “public:” sometimes and also miss , constructor, , prefix std::, deleting dynamic memory, intentionally. virtual destructor copy constructor I also consider myself a pragmatic person who wants to convey an idea in the simplest way possible rather than the standard way or using Jargons. Note: If you stumbled here directly, then I would suggest you go through first, even if it is trivial. I believe it will encourage you to explore more on this topic. What is design pattern? All of this code you encounter in this series of articles are compiled using C++20(though I have used features up to C++17 in most cases). So if you don’t have access to the latest compiler you can use which has preinstalled boost library as well. Modern C++ https://wandbox.org/ Intent To create/instantiate complex & complicated object piecewise & succinctly by providing an API in a separate entity. Builder Design Pattern is used when we want to construct a complex object. However, we do not want to have a complex constructor member or one that would need many arguments. The Builder Design Pattern constructs a complex object step by step & the final step will return the object. The process of constructing an object should be generic so that it can be used to create different representations of the same object with the help of a variety of methods. Life Without Builders Suppose you have to create the HTML generator using C++ then very naïve way to do it is : text = ; output;
output += ;
output += text;
output += ; ( , text); words[] = { , }; oss;
oss << ; ( w : words)
 oss << << w << ;
oss << ; (oss.str().c_str()); // hello auto "hello" string "" "" printf "%s" // <ul><li>hello</li><li>world</li></ul> string "hello" "world" ostringstream "<ul>" for auto " <li>" "</li>" "</ul>" printf A sophisticated dev will create a class with a bunch of constructor argument & method to add a child node. This is a good approach though but it may complicate the object representation. In general, some objects are simple & can be created in a single constructor call while other objects require a lot of ceremonies to create. Having an object with 10 constructor arguments is not productive. Instead, we should opt for piecewise construction. Builder provides an API for constructing an object step-by-step without revealing actual object representation. Builder Design Pattern Example in Modern C++ m_name; m_text; <HtmlElement> m_childs; m_indent_size = ;

 HtmlElement() = ;
 HtmlElement( &name, &text) : m_name(name), m_text(text) {} : { oss;
 oss << (m_indent_size * indent, ) << << m_name << << ; (m_text.size()) oss << (m_indent_size * (indent + ), ) << m_text << ; ( &element : m_childs)
 oss << element.str(indent + );

 oss << (m_indent_size * indent, ) << << m_name << << ; oss.str();
 } <HtmlBuilder> build( root_name) { make_unique<HtmlBuilder>(root_name); }
}; HtmlElement m_root; :
 HtmlBuilder( root_name) { m_root.m_name = root_name; } {
 m_root.m_childs.emplace_back(HtmlElement{child_name, child_text}); ;
 } { m_root.str(); } { m_root; }
}; { builder = HtmlElement::build( );
 builder->add_child( , )->add_child( , ); << builder->str() << ; EXIT_SUCCESS;
} ; class HtmlBuilder { class HtmlElement string string vector constexpr static size_t 4 default const string const string friend ; class HtmlBuilder public string str ( indent = ) int32_t 0 ostringstream string ' ' "<" ">" endl if string 1 ' ' endl for auto 1 string ' ' "</" ">" endl return static unique_ptr string return { class HtmlBuilder public string HtmlBuilder * add_child ( child_name, child_text) string string return this string str () return operator HtmlElement () return int main () auto "ul" "li" "hello" "li" "world" cout endl return /*
<ul>
 <li>
 hello
 </li>
 <li>
 world
 </li>
</ul>
*/ We are forcing users here to use builder by making data members of HtmlElements private. As you can see, we have declared the HtmlBuilder & HtmlElement in the same file & to do so, we need forward declaration i.e. class HtmlBuilder; as it is an . And we can not create the object of incomplete type before compiler parses its actual declaration. The reason is simple, the compiler needs the size of an object to allocate memory for it. Hence pointer is only way around so we have taken <HtmlBuilder>. incomplete type unique_ptr Sophisticated & Fluent Builder Design Pattern Example Following is the more sophisticated example of the Builder Design Pattern in C++ organized in four different files (i.e. Person.h, Person.cpp, PersonBuilder.h PersonBuilder.cpp). Person.h ; :: m_name, m_street_address, m_post_code, m_city; :: m_company_name, m_position, m_annual_income; Person( :: name) : m_name(name) {} : ostream& <<(ostream& os, Person& obj); ;
}; # once pragma # include <iostream> using namespace std ; class PersonBuilder { class Person std string // Personal Detail std string // Employment Detail std string public friend ; class PersonBuilder friend operator const PersonBuilder static create ( :: name) std string Person.cpp PersonBuilder Person::create( name) { PersonBuilder{name}; }

ostream& <<(ostream& os, Person& obj)
{ os << obj.m_name
 << :: << << :: << << obj.m_street_address
 << << obj.m_post_code
 << << obj.m_city
 << :: << << :: << << obj.m_company_name
 << << obj.m_position
 << << obj.m_annual_income;
} # include <iostream> # include "Person.h" # include "PersonBuilder.h" string return operator const return std endl "lives : " std endl "at " " with postcode " " in " std endl "works : " std endl "with " " as a " " earning " As you can see from the above example Person may have many details like Personal & Professional. And so does the count of data members. In our case, there are 7 data members. Having a single class for all the actions needed to create a Person through constructor might make our class bloated & lose its original purpose. Moreover, the library user needs to take care of all those constructor parameters sequence. PersonBuilder.h Person person; :
 PersonBuilder( name) : person(name) {} { move(person); } ; ; ; ; ; ; ; ;
}; # once pragma # include "Person.h" { class PersonBuilder public string operator Person () const return PersonBuilder& lives () PersonBuilder& at ( :: street_address) std string PersonBuilder& with_postcode ( :: post_code) std string PersonBuilder& in ( :: city) std string PersonBuilder& works () PersonBuilder& with ( company_name) string PersonBuilder& as_a ( position) string PersonBuilder& earning ( annual_income) string PersonBuilder.cpp PersonBuilder& PersonBuilder::lives() { * ; }

PersonBuilder& PersonBuilder::works() { * ; }

PersonBuilder& PersonBuilder::with( company_name) {
 person.m_company_name = company_name; * ;
}

PersonBuilder& PersonBuilder::as_a( position) {
 person.m_position = position; * ;
}

PersonBuilder& PersonBuilder::earning( annual_income) {
 person.m_annual_income = annual_income; * ;
}

PersonBuilder& PersonBuilder::at( :: street_address) {
 person.m_street_address = street_address; * ;
}

PersonBuilder& PersonBuilder::with_postcode( :: post_code) {
 person.m_post_code = post_code; * ;
}

PersonBuilder& PersonBuilder::in( :: city) {
 person.m_city = city; * ;
} # include "PersonBuilder.h" return this return this string return this string return this string return this std string return this std string return this std string return this Rather stuffing all those construction related APIs into Person, we can delegate that task to separate entity i.e. PersonBuilder. Main.cpp ; {
 Person p = Person::create( )
 .lives()
 .at( )
 .with_postcode( )
 .in( )
 .works()
 .with( )
 .as_a( )
 .earning( ); < # include "Person.h" # include "PersonBuilder.h" using namespace std int main () "John" "123 London Road" "SW1 1GB" "London" "PragmaSoft" "Consultant" "10e6" cout endl return Isn’t the above construction looks more intuitive, natural & plain English? If you are concerned about blank methods like lives() & works(), then do not worry, it will be eliminated in optimization. You can also observe that we are forcing user’s to use builder rather than constructor by making the constructor private & exposing only create std::string name API. ( ) Do not over complicate things by designing an interface or abstract classes unless you need it. I have seen this in many Builder Design Pattern Examples on Web. Benefits of Builder Design Pattern The number of lines of code increases at least to double in builder pattern. But the effort pays off in terms of design flexibility, fewer or no parameters to the constructor and much more readable code. Builder Design Pattern also helps in minimizing the number of parameters in constructor & thus there is no need to pass in for optional parameters to the constructor. null Immutable objects can be built without much complex logic in the object building process. Segregating construction from makes the object representation slice & precise. Having a builder entity separate provides the flexibility of creating & instantiating different objects representations. object representation Summary by FAQs When should the Builder Design Pattern be used? Whenever creation of new object requires setting many parameters and some of them (or all of them) are optional. Why do we need a Builder class when implementing a Builder Design Pattern? It isn’t necessary but there are some benefits in doing so: The concern of building object should be in the separate entity as per . SRP The original object would not be bloated. Easy & maintainable code. Testing & understanding a constructor with many input arguments gets exponentially more complicated. Greatest Advantage of Builder Design Pattern! More expressive code. MyClass o = new MyClass(5, 5.5, 'A', var, 1000, obj9, "hello"); Instead MyClass o = MyClass.builder().a(5).b(5.5).c('A').d(var).e(1000).f(obj9).g("hello"); You can see which data member is being assigned by what & even change the order of assignment. What’s the difference between Abstract Factory and Builder Design Pattern? Factory produces the objects in wholesale that could be any object from inheritance hierarchy(like Point, Point2D, Point3D). While Builder deals with instantiation of an object that is limited to a single object(Although this statement is still debatable). You see Factory is all about wholesale object creation while the builder is piecewise object creation. In both the patterns, you can separate out the mechanism related to object creation in other classes. Previously published at http://www.vishalchovatiya.com/builder-design-pattern-in-modern-cpp/

Design Patterns: Builder Pattern in Modern C++

Patrones de diseño: patrón de constructor en C++ moderno

In software engineering, Creational Design Patterns deal with object creation mechanisms, i.e. try to create objects in a manner suitable to the situation. In addition to this basic or ordinary form of object creation could result in design problems or added complexity to the design. Factory Design Pattern in C++ helps to mitigate this issue by . creating objects using separate methods or polymorphic classes By the way, If you haven’t check out my other articles on Creational Design Patterns, then here is the list: Factory Builder Prototype Singleton The code snippets you see throughout this series of articles are simplified not sophisticated. So you often see me not using keywords like override, final, public(while inheritance) just to make code compact & consumable (most of the time) in single standard screen size. I also prefer struct instead of class just to save line by not writing “public:” sometimes and also miss , constructor, , prefix std::, deleting dynamic memory, intentionally. virtual destructor copy constructor I also consider myself a pragmatic person who wants to convey an idea in the simplest way possible rather than the standard way or using Jargons. Note: If you stumbled here directly, then I would suggest you go through first, even if it is trivial. I believe it will encourage you to explore more on this topic. What is design pattern? All of this code you encounter in this series of articles are compiled using C++20(though I have used features up to C++17 in most cases). So if you don’t have access to the latest compiler you can use which has preinstalled boost library as well. Modern C++ https://wandbox.org/ Intent For the creation of wholesale objects unlike builder(which creates piecewise). Motivation Let say you have a Point class having x & y as co-ordinates which can be Cartesian or Polar coordinate as below: Point( x, y){ } }; { struct Point float float /*...*/ // Cartesian co-ordinates // Not OK: Cannot overload with same type of arguments // Point(float a, float b){ /*...*/ } // Polar co-ordinates // ... Implementation This isn't possible as you might know you can not create two constructors with the same type of arguments. Other way around is: cartesian, polar }; Point( a, b, PointTypetype = PointType::cartesian) { (type == PointType::cartesian) {
 x = a; b = y;
 } {
 x = a * (b);
 y = a * (b);
 }
 }
}; enum { class PointType { class Point float float if else cos sin But this isn't a sophisticated way of doing this. Rather we should delegate separate instantiation to separate methods. Factory Design Pattern Examples in C++ So as you can guess. We are going to mitigating constructor limitation by moving the initialization process from constructor to other structure. And we gonna be using the Factory Method for that. And just as the name suggests it uses the method or member function to initialize the object. Factory Method cartesian, polar }; m_x; m_y;
 PointType m_type; Point( x, y, PointType t) : m_x{x}, m_y{y}, m_type{t} {} : ostream & <<(ostream &os, Point &obj) { os << << obj.m_x << << obj.m_y;
 } { {x, y, PointType::cartesian};
 } { {a * (b), a * (b), PointType::polar};
 }
}; { p = Point::NewPolar( , M_PI_4); << p << ; EXIT_SUCCESS;
} enum { class PointType { class Point float float // Private constructor, so that object can't be created directly const float const float public friend operator const return "x: " " y: " Point static NewCartesian ( x, y) float float return Point static NewPolar ( a, b) float float return cos sin int main () // Point p{ 1,2 }; // will not work auto 5 cout endl // x: 3.53553 y: 3.53553 return As you can observe from the implementation. It actually disallows the use of constructor & forcing users to use static methods instead. And this is the essence of the . Factory Method i.e. private constructor & static method Classical Factory Design Pattern If you have dedicated code for construction then while not we move it to a dedicated class. And Just to do separation of concerns i.e. from SOLID design principles. Single Responsibility Principle }; : { { x, y };
 } { { r* (theta), r* (theta) };
 }
}; { class Point // ... as it is from above friend ; class PointFactory { class PointFactory public Point static NewCartesian ( x, y) float float return Point static NewPolar ( r, theta) float float return cos sin Mind that this is not the abstract factory this is a concrete factory. Making the PointFactory friend class of Point we have violated the (OCP). As friend keyword itself contrary to OCP. Open-Closed Principle Inner Factory There is a critical thing we missed in our Factory that there is no strong link between PointFactory & Point which confuses user to use Point just by seeing everything is private. So rather than designing a factory outside the class. We can simply put it in the class which encourage users to use Factory. Thus, we also serve the second problem which is breaking the . And this will be somewhat more intuitive for the user to use Factory. Open-Closed Principle m_x; m_y;

 Point( x, y) : m_x(x), m_y(y) {} : { { x,y }; } { { r* (theta), r* (theta) }; }
 };
}; { p = Point::Factory::NewCartesian( , ); EXIT_SUCCESS;
} { class Point float float float float public { struct Factory Point static NewCartesian ( x, y) float float return Point static NewPolar ( r, theta) float float return cos sin int main () auto 2 3 return Abstract Factory do we need an Abstract Factory? Why C++ has the support of polymorphic object destruction using it’s base class’s . Similarly, equivalent support for creation & copying of objects is missing as С++ doesn’t support & . virtual destructor virtual constructor copy constructors Moreover, you can’t create an object unless you know its static type, because the compiler must know the amount of space it needs to allocate. For the same reason, copy of an object also requires its type to known at compile-time. ~Point(){ << ; }
}; Point {
 ~Point2D(){ << ; }
}; Point {
 ~Point3D(){ << ; }
}; { who; } { struct Point virtual cout "~Point\n" : struct Point2D cout "~Point2D\n" : struct Point3D cout "~Point3D\n" void who_am_i (Point *who) // Not sure whether Point2D would be passed here or Point3D // How to `create` the object of same type i.e. pointed by who ? // How to `copy` object of same type i.e. pointed by who ? delete // you can delete object pointed by who, thanks to virtual destructor Example of Abstract Factory Design Pattern The Abstract Factory is useful in a situation that requires the creation of many different types of objects, all derived from a common base type. The Abstract Factory defines a method for creating the objects, which can then override to specify the derived type that will be created. Thus, at run time, the appropriate Abstract Factory Method will be called depending upon the type of object referenced/pointed & return a base class pointer to a new instance of that object. subclasses ~Point() = ; <Point> create() = ; <Point> clone() = ;
}; Point { <Point> create() { make_unique<Point2D>(); } <Point> clone() { make_unique<Point2D>(* ); }
}; Point { <Point> create() { make_unique<Point3D>(); } <Point> clone() { make_unique<Point3D>(* ); }
}; { new_who = who->create(); duplicate_who = who->clone(); who;
} { struct Point virtual default virtual unique_ptr 0 virtual unique_ptr 0 : struct Point2D unique_ptr return unique_ptr return this : struct Point3D unique_ptr return unique_ptr return this void who_am_i (Point *who) auto // `create` the object of same type i.e. pointed by who ? auto // `copy` the object of same type i.e. pointed by who ? delete As shown above, we have leveraged polymorphic methods by delegating the act of creation & copying the object to the derived class through the use of pure virtual methods. Above code is not only implement (i.e. create()) but also implements (i.e. clone()). virtual constructor virtual copy constructor Make sure while using Abstract Factory you have ensured the . Liskov's Substitution Principle(LSP) Functional Approach to Factory Design Pattern using Modern C++ In our Abstract Factory example, we have followed the object-oriented approach but its equally possible nowadays to a more functional approach. So, let's build a similar kind of Factory without relying on polymorphic functionality as it might not suit some time-constrained application like an . Because the may troll system during critical functionality. embedded system virtual table & dynamic dispatch mechanism This is pretty straight forward as it uses functional & as follows: lambda functions }; Point { }; Point { }; <PointType, function< <Point>() >> m_factories; :
 PointFunctionalFactory() {
 m_factories[PointType::Point2D] = [] { make_unique<Point2D>(); };
 m_factories[PointType::Point3D] = [] { make_unique<Point3D>(); };
 } <Point> create(PointType type) { m_factories[type](); } 
}; {
 PointFunctionalFactory pf; p2D = pf.create(PointType::Point2D); EXIT_SUCCESS;
} { struct Point /* . . . */ : struct Point2D /* . . . */ : struct Point3D /* . . . */ { class PointFunctionalFactory map unique_ptr public return return unique_ptr return int main () auto return If you are thinking that we are over-engineering, then keep in mind that our object construction is simple here just to demonstrate the technique & so does our lambda function. When your object representation increases, it requires a lot of methods to call in order to instantiate object properly, in such case you just need to modify lambda expression of the factory or introduce . Builder Design Pattern Benefits of Factory Design Pattern Single point/class for different object creation. Thus easy to maintain & understand software. You can create the object without even knowing its type by using Abstract Factory. It provides great modularity. Imagine programming a video game, where you would like to add new types of enemies in the future, each of which has different AI functions and can update differently. By using a factory method, the controller of the program can call to the factory to create the enemies, without any dependency or knowledge of the actual types of enemies. Now, future developers can create new enemies, with new AI controls and new drawing member functions, add it to the factory, and create a level which calls the factory, asking for the enemies by name. Combine this method with an XML description of levels, and developers could create new levels without having to recompile their program. All this, thanks to the separation of creation of objects from the usage of objects. Allows you to change the design of your application more readily, this is known as loose coupling. Summary by FAQs Abstract Factory & Functional Factory is always a good choice. What is the correct way to implement the Factory Design Pattern in C++? Factory vs Abstract Factor vs Functional Factory? Factory: Create an object with varied instantiation. Abstract Factor: Create an object without knowing its type & refer using base class pointer & reference. Access using polymorphic methods. Functional Factory: When object creation is more complex. Abstract Factory + . Although I have not included Builder in Functional Factory example. Builder Design Pattern What's the difference between Abstract Factory and Builder Design Pattern? Factory produces the objects in wholesale that could be any object from inheritance hierarchy(like Point, Point2D, Point3D). While Builder deals with instantiation of an object that is limited to a single object(Although this statement is still debatable). You see Factory is all about wholesale object creation while the builder is piecewise object creation. In both the patterns, you can separate out the mechanism related to object creation in other classes. When to use the Factory Design Pattern? Employ Factory Design Pattern to create an object of required functionality(s) but type of object will remain undecided or it will be decided ob dynamic parameters being passed. Previously published at http://www.vishalchovatiya.com/factory-design-pattern-in-modern-cpp/

Design Patterns: Exploring Factory Method in Modern C++

Patrones de diseño: exploración del método de fábrica en C++ moderno

Imagine there is something wrong with the water pressure in your house. You have to call to the plumber service. They say that there is a guy, who is honest, and capable, and has a Ph. D. in plumbing, but… He doesn’t speak English well. He came from Tajikistan and only speaks fluent Tajik. But it just so happens that you studied Tajik in college, so you say that they send the guy immediately. When he arrives, you describe the problem (in Tajik), and he gets to work. Soon enough he finds the problem and tries to report it to you, apparently waiting for a reaction. But for some reason he doesn’t use Tajik to report it, he uses English. Which… He doesn’t really know. If you reply (in Tajik) that you don’t understand, he will not switch to Tajik; he will only repeat the same phrase louder. Which doesn’t help to build a dialog at all. Nobody knows why he consistently chooses to reply in the language he doesn’t speak well instead of his own. That you do. It is absurdly irritating, but that’s just how the things work. It’s how they always been. What he speaks in is basically a bad English mixed with professional jargonisms and Tajik obscenities. It looks something like this: In function ‘int main()’: 19:16: error: cannot bind ‘std::ostream {aka std::basic_ostream<char>}’ lvalue to ‘std::basic_ostream<char>&&’ In file included from /usr/include/c++/4.9/iostream:39:0, from 2: /usr/include/c++/4.9/ostream:602:5: note: initializing argument 1 of ‘std::basic_ostream<_CharT, _Traits>& std::operator<<(std::basic_ostream<_CharT, _Traits>&&, const _Tp&) [with _CharT = char; _Traits = std::char_traits<char>; _Tp = Element]’ operator<<(basic_ostream<_CharT, _Traits>&& __os, const _Tp& __x) Ok, it isn’t about plumbing anymore. In fact, It never was about plumbing. It’s about how we communicate with the machine. A programming language is something we use to tell it what to do, but the machine tries to speak back in English when something gets wrong. Computers don’t speak English well, although they are good at processing formal languages. And presumably, the one who writes in formal language should be able to read it back with no problem. Wouldn’t it be better, if the compiler or interpreter would respond in the same language it gets its commands in? Somehow the idea seems not all that agreeable without a Tajik plumber. So far the only language I know that reports errors in the same language you write code is Erlang. If you know more, please let me know in the comment section below. Here’s a simple example from “ ”: Programming Erlang 1> Point = {point, 10, 45}.
2> {point, C, C} = Point.
=ERROR REPORT==== 28-Oct-2006::17:17:00 ===
Error in process <0.32.0> with exit value:
{{badmatch,{point,10,45}},[{erl_eval,expr,3}]} The string in bold is the error message, the rest is just a preamble from the interpreter. Indeed, it isn’t entirely convenient for the casual reader. You have to know Erlang to read its errors. But since you still need to know Erlang to write the code that causes them, it’s fine. The alternatives are much worse. With informal and non-standardized error messaging you would have to know not only the programming language itself but its error messaging Pidgin as well. Every C++ practitioner can confirm that reading STL messages is an art by itself. It takes knowledge, patience, determination and, occasionally, a crystal ball to build a firm understanding of a reported problem. For instance, the error message from above reads as “sorry, I can’t print out an element for you, because its type doesn’t have a << operator.” Normally the language is something you can build a dialog in. Programming languages are an unlucky exemption. Historically it was more important to tell a computer what to do rather than the opposite. When you write a small single-threaded program you can hope that there will be no errors whatsoever, so you don’t really need a reporting at all. Error reporting was always viewed as an auxiliary feature. Erlang, however, aims at building highly complex distributed systems. Errors are not only most likely to happen, they are inherent to the nature of the domain. Everything that could go wrong eventually will. And you better have a decent reporting language for that. The very fact that you can have a dialog, meaning writing code and receiving errors in the same language, makes Erlang not only programming but a true computer language.

Why is Erlang the only true computer language

Setting up Data Streams Using NiPyAPI: UI-Free Guide

Let's build a chatbot with just Python using the Streamlit library, Ollama, and Microsoft Phi-3. Streamlit Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience required. You can find more info in the official Streamlit docs. Ollama Ollama allows you to run open-source large language models, locally. You can find more info in the official Ollama docs. Phi-3 Mini Phi-3 Mini is a 3.8B parameters, lightweight, state-of-the-art open model by Microsoft. You can find more info in the official Phi-3 Mini docs. Steps Create a new conda environment conda create --name envStreamPhi Activate the environment conda activate envStreamPhi Install ollama Download Ollama from https://ollama.com/download pull the phi-3 model The model we will be using is located here: ollama pull phi3 Pull the Embeddings model: ollama pull nomic-embed-text Test installation streamlit hello Build the AI Assistant In order to build the AI assistant, you have 2 choices: clone the repo and get all the code from the get-go, or code along with me. I - First option: Clone the project from GitHub git clone https://github.com/JalelTounsi/oLLaMaStreamPhi.git Run the Application streamlit run app.py II - Second Option: Code Along Create your app.py file Add Imports import streamlit as st
import ollama Add the Defacto Message if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}] Add the Message History for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"]) Configure Model def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token Configure the Prompt if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) All the Codebase of app.py import streamlit as st
import ollama

st.title("💬 Phi3 Chatbot")

if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}]

### Write Message History
for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"])

## Configure the model
def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token

if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) Run the Streamlit App streamlit run app.py Bonus Point Feel free to check this video to understand the code, follow along, and run the AI assistant locally on your computer: AI assistant using Streamlit, Phi3, and Ollama Let's build a chatbot with just Python using the Streamlit library, Ollama , and Microsoft Phi-3 . Streamlit Ollama Microsoft Phi-3 Streamlit Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience required. You can find more info in the official Streamlit docs . official Streamlit docs Ollama Ollama allows you to run open-source large language models, locally. You can find more info in the official Ollama docs . official Ollama Ollama docs Phi-3 Mini Phi-3 Mini is a 3.8B parameters, lightweight, state-of-the-art open model by Microsoft. You can find more info in the official Phi-3 Mini docs . Phi-3 Mini Phi-3 Mini docs Steps Create a new conda environment conda create --name envStreamPhi conda create --name envStreamPhi Activate the environment conda activate envStreamPhi conda activate envStreamPhi Install ollama Download Ollama from https://ollama.com/download https://ollama.com/download pull the phi-3 model The model we will be using is located here : here ollama pull phi3 ollama pull phi3 Pull the Embeddings model: ollama pull nomic-embed-text ollama pull nomic-embed-text Test installation streamlit hello streamlit hello Build the AI Assistant In order to build the AI assistant, you have 2 choices: clone the repo and get all the code from the get-go, or code along with me. I - First option: Clone the project from GitHub git clone https://github.com/JalelTounsi/oLLaMaStreamPhi.git git clone https://github.com/JalelTounsi/oLLaMaStreamPhi.git Run the Application streamlit run app.py streamlit run app.py II - Second Option: Code Along Create your app.py file app.py Add Imports import streamlit as st
import ollama import streamlit as st
import ollama Add the Defacto Message if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}] if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}] Add the Message History for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"]) for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"]) Configure Model def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token Configure the Prompt if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) All the Codebase of app.py app.py import streamlit as st
import ollama

st.title("💬 Phi3 Chatbot")

if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}]

### Write Message History
for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"])

## Configure the model
def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token

if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) import streamlit as st
import ollama

st.title("💬 Phi3 Chatbot")

if "messages" not in st.session_state:
    st.session_state["messages"] = [{"role": "assistant", "content": "Hello tehre, how can I help you, today?"}]

### Write Message History
for msg in st.session_state.messages:
    if msg["role"] == "user":
        st.chat_message(msg["role"], avatar="🧑‍💻").write(msg["content"])
    else:
        st.chat_message(msg["role"], avatar="🤖").write(msg["content"])

## Configure the model
def generate_response():
    response = ollama.chat(model='phi3', stream=True, messages=st.session_state.messages)
    for partial_resp in response:
        token = partial_resp["message"]["content"]
        st.session_state["full_message"] += token
        yield token

if prompt := st.chat_input():
    st.session_state.messages.append({"role": "user", "content": prompt})
    st.chat_message("user", avatar="🧑‍💻").write(prompt)
    st.session_state["full_message"] = ""
    st.chat_message("assistant", avatar="🤖").write_stream(generate_response)
    st.session_state.messages.append({"role": "assistant", "content": st.session_state["full_message"]}) Run the Streamlit App streamlit run app.py streamlit run app.py Bonus Point Feel free to check this video to understand the code, follow along, and run the AI assistant locally on your computer: AI assistant using Streamlit, Phi3, and Ollama AI assistant using Streamlit, Phi3, and Ollama

Build a Free-to-Use Open-Source AI Assistant With Streamlit, Microsoft Phi-3, & Ollama

I wrote previously about libs for error management in Rust. This week, I want to write about the try block, an experimental feature. The limit of the ? operator Please check the above article for a complete refresher on error management in general and the ? operator in particular. In short, ? allows it to hook into a function call that returns a Result: If the Result contains a value, it continues normally
If it contains an error, it short-circuits and returns the Result to the calling function. fn add(str1: &str, str2: &str) -> Result<i8, ParseIntError> {
 Ok(str1.parse::<i8>()? + str2.parse::<i8>()?)
}

fn main() {
 print!("{:?}", add("1", "2"));
 print!("{:?}", add("1", "a"));
} The output is the following: Ok(3)
Err(ParseIntError { kind: InvalidDigit }) Note that the defining function's signature must return a Result or an Option. The following block doesn't compile: fn add(str1: &str, str2: &str) -> i8 {
 str1.parse::<i8>()? + str2.parse::<i8>()?
} the `?` operator can only be used in a function that returns `Result` or `Option` The verbose alternative We must manually unwrap to return a non-wrapper type, e.g., i8 instead of Option<i8>. fn add(str1: &str, str2: &str) -> i8 {
 let int1 = str1.parse::<i8>(); //1
 let int2 = str2.parse::<i8>(); //1
 if int1.is_err() || int2.is_err() { -1 } //2-3
 else { int1.unwrap() + int2.unwrap() } //4
} Define Result variables


Manually checks if any of the variables contains an error, i.e., the parsing failed


Return a default value since we cannot get a Result. In this case, it's not a great idea, but it's for explanation's sake


Unwrap with confidence The try block to the rescue The sample above works but is quite lengthy. The try block is an experimental approach to make it more elegant. It allows "compacting" all the checks for errors in a single block: #![feature(try_blocks)] //1

fn add(str1: &str, str2: &str) -> i8 {
 let result = try {
 let int1 = str1.parse::<i8>();
 let int2 = str2.parse::<i8>();
 int1.unwrap()? + int2.unwrap()? //2
 };
 if result.is_err() { -1 } //3
 else { result.unwrap() } //4
} Enable the experimental feature
Use the ? operator though the defining function doesn't return Result
Check for errors only once
Unwrap confidently Alas, the code doesn't compile: the `?` operator can only be applied to values that implement `Try` i8 doesn't implement Try. Neither i8 nor Try belong to our crate; a custom implementation would require the use of the wrapper-type pattern. Fortunately, a couple of types already implement Try: Result, Option, Poll, and ControlFlow. fn add(str1: &str, str2: &str) -> i8 {
 let result: Result<i8, ParseIntError> = try { //1
 str1.parse::<i8>()? + str2.parse::<i8>()? //2
 };
 if result.is_err() { -1 }
 else { result.unwrap() }
} The compiler cannot infer the type
Using ? on Result inside the try block is now allowed Conclusion I learned about the try block in Java over twenty years ago. Java needs it because exceptions are at the root of its error-handling system; Rust doesn't because it uses Functional Programming for its error handling - mainly Result. The ? operator builds upon the Result type to allow short-circuiting in functions that return Result themselves. If the function doesn't, you need a lot of boilerplate code. The experimental try block relieves some of it. To go further: Error management in Rust, and libs that support it
"The Rust Unstable Book: try_blocks"
The Rust RFC Book
Extending Rust's Effect System I wrote previously about libs for error management in Rust . This week, I want to write about the try block, an experimental feature. libs for error management in Rust try The limit of the ? operator ? Please check the above article for a complete refresher on error management in general and the ? operator in particular. In short, ? allows it to hook into a function call that returns a Result : ? ? Result If the Result contains a value, it continues normally If it contains an error, it short-circuits and returns the Result to the calling function. If the Result contains a value, it continues normally Result If it contains an error, it short-circuits and returns the Result to the calling function. Result fn add(str1: &str, str2: &str) -> Result<i8, ParseIntError> {
 Ok(str1.parse::<i8>()? + str2.parse::<i8>()?)
}

fn main() {
 print!("{:?}", add("1", "2"));
 print!("{:?}", add("1", "a"));
} fn add(str1: &str, str2: &str) -> Result<i8, ParseIntError> {
 Ok(str1.parse::<i8>()? + str2.parse::<i8>()?)
}

fn main() {
 print!("{:?}", add("1", "2"));
 print!("{:?}", add("1", "a"));
} The output is the following: Ok(3)
Err(ParseIntError { kind: InvalidDigit }) Ok(3)
Err(ParseIntError { kind: InvalidDigit }) Note that the defining function's signature must return a Result or an Option . The following block doesn't compile: must Result Option fn add(str1: &str, str2: &str) -> i8 {
 str1.parse::<i8>()? + str2.parse::<i8>()?
} fn add(str1: &str, str2: &str) -> i8 {
 str1.parse::<i8>()? + str2.parse::<i8>()?
} the `?` operator can only be used in a function that returns `Result` or `Option` the `?` operator can only be used in a function that returns `Result` or `Option` The verbose alternative We must manually unwrap to return a non-wrapper type, e.g. , i8 instead of Option<i8> . e.g. i8 Option<i8> fn add(str1: &str, str2: &str) -> i8 {
 let int1 = str1.parse::<i8>(); //1
 let int2 = str2.parse::<i8>(); //1
 if int1.is_err() || int2.is_err() { -1 } //2-3
 else { int1.unwrap() + int2.unwrap() } //4
} fn add(str1: &str, str2: &str) -> i8 {
 let int1 = str1.parse::<i8>(); //1
 let int2 = str2.parse::<i8>(); //1
 if int1.is_err() || int2.is_err() { -1 } //2-3
 else { int1.unwrap() + int2.unwrap() } //4
} Define Result variables Manually checks if any of the variables contains an error, i.e., the parsing failed Return a default value since we cannot get a Result. In this case, it's not a great idea, but it's for explanation's sake Unwrap with confidence Define Result variables Define Result variables Result Manually checks if any of the variables contains an error, i.e., the parsing failed Manually checks if any of the variables contains an error, i.e. , the parsing failed i.e. Return a default value since we cannot get a Result. In this case, it's not a great idea, but it's for explanation's sake Return a default value since we cannot get a Result . In this case, it's not a great idea, but it's for explanation's sake Result Unwrap with confidence Unwrap with confidence The try block to the rescue try The sample above works but is quite lengthy. The try block is an experimental approach to make it more elegant. It allows "compacting" all the checks for errors in a single block: try experimental #![feature(try_blocks)] //1

fn add(str1: &str, str2: &str) -> i8 {
 let result = try {
 let int1 = str1.parse::<i8>();
 let int2 = str2.parse::<i8>();
 int1.unwrap()? + int2.unwrap()? //2
 };
 if result.is_err() { -1 } //3
 else { result.unwrap() } //4
} #![feature(try_blocks)] //1

fn add(str1: &str, str2: &str) -> i8 {
 let result = try {
 let int1 = str1.parse::<i8>();
 let int2 = str2.parse::<i8>();
 int1.unwrap()? + int2.unwrap()? //2
 };
 if result.is_err() { -1 } //3
 else { result.unwrap() } //4
} Enable the experimental feature Use the ? operator though the defining function doesn't return Result Check for errors only once Unwrap confidently Enable the experimental feature Use the ? operator though the defining function doesn't return Result ? Result Check for errors only once Unwrap confidently Alas, the code doesn't compile: the `?` operator can only be applied to values that implement `Try` the `?` operator can only be applied to values that implement `Try` i8 doesn't implement Try . Neither i8 nor Try belong to our crate; a custom implementation would require the use of the wrapper-type pattern. Fortunately, a couple of types already implement Try : Result , Option , Poll , and ControlFlow . i8 Try i8 Try Try Result Option Poll ControlFlow fn add(str1: &str, str2: &str) -> i8 {
 let result: Result<i8, ParseIntError> = try { //1
 str1.parse::<i8>()? + str2.parse::<i8>()? //2
 };
 if result.is_err() { -1 }
 else { result.unwrap() }
} fn add(str1: &str, str2: &str) -> i8 {
 let result: Result<i8, ParseIntError> = try { //1
 str1.parse::<i8>()? + str2.parse::<i8>()? //2
 };
 if result.is_err() { -1 }
 else { result.unwrap() }
} The compiler cannot infer the type Using ? on Result inside the try block is now allowed The compiler cannot infer the type Using ? on Result inside the try block is now allowed ? Result try Conclusion I learned about the try block in Java over twenty years ago. Java needs it because exceptions are at the root of its error-handling system; Rust doesn't because it uses Functional Programming for its error handling - mainly Result . try Result The ? operator builds upon the Result type to allow short-circuiting in functions that return Result themselves. If the function doesn't, you need a lot of boilerplate code. The experimental try block relieves some of it. ? Result Result try To go further: To go further: Error management in Rust, and libs that support it "The Rust Unstable Book: try_blocks" The Rust RFC Book Extending Rust's Effect System Error management in Rust, and libs that support it Error management in Rust, and libs that support it "The Rust Unstable Book: try_blocks" "The Rust Unstable Book: try_blocks" The Rust RFC Book The Rust RFC Book Extending Rust's Effect System Extending Rust's Effect System

The Try Block in Rust: Streamlining Error Management Beyond the '?' Operator

Real-Time tricks: Harnessing Kafka Streams for Seamless Data Tasks

Today’s business world has become very appreciative of agility and efficiency. Companies keep searching for new ways to optimize working processes and intensify the productivity of said processes. One of these ways is low-code development. This article dives into how low-code development can be helpful in business process automation and the upsurge of process transparency, better guaranteeing business stability and scalability What is Low-Code? Low code is an application development approach that speeds up software delivery by minimizing manual coding to the largest extent possible. It leverages visual development tools and pre-built templates to help business owners, process owners, and low-coding professionals develop and deploy applications in no time. Key Benefits for Low-Code for Businesses **Rapid Development & Deployment
\ With Low-Code platforms, businesses can build and deploy applications quickly while easily being able to respond to the needs of business for at least better and quicker internal processes.


**Process Optimization & Automation
\ By automating routine tasks and streamlining workflows, companies can enhance efficiency, reduce human error, and free up employees to focus on more strategic work.


**Increased Business Agility
\ The ease and speed of developing applications with low-code platforms allow businesses to experiment with new ideas and strategies without significant investment, fostering innovation and adaptability. Low-Code in Action: Transforming Internal Processes of Antifraud Team The anti-fraud process acts as a cornerstone for the identification and mitigation of fraudulent activities among clients, employees, and partners. Its key objective is to decrease material financial loss, improve business operation integrity, and increase efficiency through the detection of fraud schemes, correction of staff errors, and improvement of inner policy Initially, this had been in place using Excel sheets, and everything was to be manually distributed between the two members, which sufficed for a small team. With the process scope increasing, it was identified that Excel dependence was not enough. However, Excel is limited, and this results in operational inefficiencies, lack of scalability, lower productivity, and more errors. It would make the whole process opaque and cumbersome for onboarding new staff, integrating new features, and being able to experiment easily To confront these challenges and make the process even on a bigger scale, a more sophisticated approach was necessary. The key objectives were to increase visibility, enable the easy tracking and measuring of the effectiveness of the process, identification of areas for improvement, and make rapid changes associated with business growth and changing requirements. This required transitioning from Excel to a system that would meet these goals much more flexibly. A low-code platform was able to help them create a new tool, and that is what made the antifraud process significantly more efficient. They have been able to help develop a product with clear processes that have enabled them to extend to about 20 team members. The new system easily identifies and resolves issues, quickly integrates new functionality, and responds agilely to the ever-fluctuating business and market needs Variants for implementation What could be used: DWH (or another source of business data)— to aggregate all necessary data for task creation and to display information to users;


N8N — to fetch data from the DWH and automate the task creation process, streamlining operations and ensuring that tasks are generated based on the latest data insights;


Appsmith — to develop the front-end interface, enabling a user-friendly environment for interaction with the system;


PostgreSQL — serves as the application’s database, offering a robust and secure environment for storing application data, user decisions, and investigation artifacts. The system generates and presents a list of tasks to the fraud team members based on specific triggers automatically
When a team member decides on a task, there is a display of all necessary pieces of information reflecting on the current status of the case that gives the possibility to make their research. Following this, the user can make informed decisions, saving these along with any evidence or artifacts from their investigation within the tool
Then, the results are further stored securely and, in an effective way, shared with the related processes that might be impacted by the decisions of the anti-fraud team, and these include accounting, HR, business teams, and others The whole system, complete with all integrations and testing, can be set up in a matter of weeks, allowing for continuous improvement and adaptation to changing business needs Considerations While there are many advantages of low-code development, there are still some things to consider, such as the importance of traditional software development practices like CI/CD, code reviews, functional testing, regression testing, version control, alerting, monitoring, and so on. However, there are a small number of diverse tools which can fully implement this process. As such, you have to make your own decision for the implementation of these processes. By the way, these processes are very important. Because low-code platforms guarantee fast change and rapid implementation, but this speed could be tricky and can further lead to issues because of a lack of thorough testing or oversight Conclusion Low-code development is becoming a game-changer in business for optimizing and automating internal processes By enabling rapid development, enhancing agility, and reducing costs, low-code platforms are becoming an essential tool in the digital transformation toolkit. However, it is crucial for businesses to carefully evaluate their specific needs, considering the potential challenges and ensuring proper testing and quality assurance and other code development practices are established The right strategy and tools put in place ensure that the low-code empowers your businesses to innovate and grow Tg: https://t.me/rmzkv
LinkedIn: https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/ Today’s business world has become very appreciative of agility and efficiency. Companies keep searching for new ways to optimize working processes and intensify the productivity of said processes. One of these ways is low-code development. This article dives into how low-code development can be helpful in business process automation and the upsurge of process transparency, better guaranteeing business stability and scalability What is Low-Code? Low code is an application development approach that speeds up software delivery by minimizing manual coding to the largest extent possible. It leverages visual development tools and pre-built templates to help business owners, process owners, and low-coding professionals develop and deploy applications in no time. Key Benefits for Low-Code for Businesses **Rapid Development & Deployment
\ With Low-Code platforms, businesses can build and deploy applications quickly while easily being able to respond to the needs of business for at least better and quicker internal processes. **Process Optimization & Automation
\ By automating routine tasks and streamlining workflows, companies can enhance efficiency, reduce human error, and free up employees to focus on more strategic work. **Increased Business Agility
\ The ease and speed of developing applications with low-code platforms allow businesses to experiment with new ideas and strategies without significant investment, fostering innovation and adaptability. **Rapid Development & Deployment
\ With Low-Code platforms, businesses can build and deploy applications quickly while easily being able to respond to the needs of business for at least better and quicker internal processes. **Rapid Development & Deployment
\ With Low-Code platforms, businesses can build and deploy applications quickly while easily being able to respond to the needs of business for at least better and quicker internal processes. **Process Optimization & Automation
\ By automating routine tasks and streamlining workflows, companies can enhance efficiency, reduce human error, and free up employees to focus on more strategic work. **Process Optimization & Automation
\ By automating routine tasks and streamlining workflows, companies can enhance efficiency, reduce human error, and free up employees to focus on more strategic work. **Increased Business Agility
\ The ease and speed of developing applications with low-code platforms allow businesses to experiment with new ideas and strategies without significant investment, fostering innovation and adaptability. **Increased Business Agility
\ The ease and speed of developing applications with low-code platforms allow businesses to experiment with new ideas and strategies without significant investment, fostering innovation and adaptability. Low-Code in Action: Transforming Internal Processes of Antifraud Team The anti-fraud process acts as a cornerstone for the identification and mitigation of fraudulent activities among clients, employees, and partners. Its key objective is to decrease material financial loss, improve business operation integrity, and increase efficiency through the detection of fraud schemes, correction of staff errors, and improvement of inner policy Initially, this had been in place using Excel sheets, and everything was to be manually distributed between the two members, which sufficed for a small team. With the process scope increasing, it was identified that Excel dependence was not enough. However, Excel is limited, and this results in operational inefficiencies, lack of scalability, lower productivity, and more errors. It would make the whole process opaque and cumbersome for onboarding new staff, integrating new features, and being able to experiment easily To confront these challenges and make the process even on a bigger scale, a more sophisticated approach was necessary. The key objectives were to increase visibility, enable the easy tracking and measuring of the effectiveness of the process, identification of areas for improvement, and make rapid changes associated with business growth and changing requirements. This required transitioning from Excel to a system that would meet these goals much more flexibly. A low-code platform was able to help them create a new tool, and that is what made the antifraud process significantly more efficient. They have been able to help develop a product with clear processes that have enabled them to extend to about 20 team members. The new system easily identifies and resolves issues, quickly integrates new functionality, and responds agilely to the ever-fluctuating business and market needs Variants for implementation What could be used: DWH (or another source of business data)— to aggregate all necessary data for task creation and to display information to users; N8N — to fetch data from the DWH and automate the task creation process, streamlining operations and ensuring that tasks are generated based on the latest data insights; Appsmith — to develop the front-end interface, enabling a user-friendly environment for interaction with the system; PostgreSQL — serves as the application’s database, offering a robust and secure environment for storing application data, user decisions, and investigation artifacts. DWH (or another source of business data)— to aggregate all necessary data for task creation and to display information to users; DWH (or another source of business data) — to aggregate all necessary data for task creation and to display information to users; DWH (or another source of business data) (or another source of business data) N8N — to fetch data from the DWH and automate the task creation process, streamlining operations and ensuring that tasks are generated based on the latest data insights; N8N — to fetch data from the DWH and automate the task creation process, streamlining operations and ensuring that tasks are generated based on the latest data insights; N8N Appsmith — to develop the front-end interface, enabling a user-friendly environment for interaction with the system; Appsmith — to develop the front-end interface, enabling a user-friendly environment for interaction with the system; Appsmith PostgreSQL — serves as the application’s database, offering a robust and secure environment for storing application data, user decisions, and investigation artifacts. PostgreSQL — serves as the application’s database, offering a robust and secure environment for storing application data, user decisions, and investigation artifacts. PostgreSQL The system generates and presents a list of tasks to the fraud team members based on specific triggers automatically When a team member decides on a task, there is a display of all necessary pieces of information reflecting on the current status of the case that gives the possibility to make their research. Following this, the user can make informed decisions, saving these along with any evidence or artifacts from their investigation within the tool Then, the results are further stored securely and, in an effective way, shared with the related processes that might be impacted by the decisions of the anti-fraud team, and these include accounting, HR, business teams, and others The system generates and presents a list of tasks to the fraud team members based on specific triggers automatically When a team member decides on a task, there is a display of all necessary pieces of information reflecting on the current status of the case that gives the possibility to make their research. Following this, the user can make informed decisions, saving these along with any evidence or artifacts from their investigation within the tool Then, the results are further stored securely and, in an effective way, shared with the related processes that might be impacted by the decisions of the anti-fraud team, and these include accounting, HR, business teams, and others The whole system, complete with all integrations and testing, can be set up in a matter of weeks, allowing for continuous improvement and adaptation to changing business needs Considerations While there are many advantages of low-code development, there are still some things to consider, such as the importance of traditional software development practices like CI/CD, code reviews, functional testing, regression testing, version control, alerting, monitoring, and so on. However, there are a small number of diverse tools which can fully implement this process. As such, you have to make your own decision for the implementation of these processes. By the way, these processes are very important. Because low-code platforms guarantee fast change and rapid implementation, but this speed could be tricky and can further lead to issues because of a lack of thorough testing or oversight Conclusion Low-code development is becoming a game-changer in business for optimizing and automating internal processes By enabling rapid development, enhancing agility, and reducing costs, low-code platforms are becoming an essential tool in the digital transformation toolkit. However, it is crucial for businesses to carefully evaluate their specific needs, considering the potential challenges and ensuring proper testing and quality assurance and other code development practices are established The right strategy and tools put in place ensure that the low-code empowers your businesses to innovate and grow Tg: https://t.me/rmzkv LinkedIn: https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/ Tg: https://t.me/rmzkv Tg: https://t.me/rmzkv https://t.me/rmzkv https://t.me/rmzkv LinkedIn: https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/ LinkedIn: https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/ https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/ https://www.linkedin.com/in/ramiz-zakiev-8b77791a7/

Using Low-Code Development to Streamline Internal Business Processes

Essential Entity Framework Core Tips: How to Optimize Performance, Streamline Queries, and More

Authors:
(1) Vinod S. Khandkar and Manjesh K. Hanawal, Industrial Engineering and Operations Research Indian Institute of Technology Bombay, Mumbai, India and {vinod.khandkar, mhanawal}@iitb.ac.in. Table of Links Abstract & Introduction Related Work and Background Challenges in TD Detection Measurement Setup Development Case Study : Wehe - TD Detection Tool for Mobile Environment Shortcoming of Wehe on HTTPS Traffic TD Detection of HTTPS Traffic Conclusion & References II. RELATED WORK AND BACKGROUND As net-neutrality regulations were hotly debated in many countries, several authors have proposed methods to detect any net neutrality violations or deliberate traffic discriminations in the Internet. Recent surveys [1], [2] cover all the tools and techniques used by them. The literature is also rich with various forms of discrimination and its detection, like discrimination of content providers [3], end-users [4], specific services like BitTorrent [5]. Our focus in this work is on detecting discrimination of streaming services (both audio and video) that constitute a major portion of internet traffic and consume significant bandwidth. We discuss some of the tools and techniques that is most relevant to our work. A. Existing TD detection tools There are two commonly used techniques for detecting TD in users’ Internet traffic. One type of approach passively monitors traffic [6]. In such cases, the end-result or TD result is not immediately available to the user. Instead, the tool provides the aggregated result of traffic differentiation over the given ISP. Another type of detection technique uses specially crafted probing traffic - called Active probes. It analyses network response to probing traffic to detect any anomaly. [6]–[10] describes measurement setups based on such active probing. It uses traffic parameters such as packet loss, latency, packet sequence, or pattern to identify network operation characteristics or detect anomalies. Some tool uses multiple types of probing traffics called active differential probes. While one probing traffic type undergoes standard network middle-box processing, the other probing traffic type is supposed to evade any traffic differentiation by being not amenable to middle-box processing. Typically, in such differential probe methods, one traffic type mimics characters of traffic of services accessed from the original server (like Youtube, Netflix). The other is a reference or control traffic. In this method, network responses to both traffic streams are compared to check if performance of any stream suffered against the other. [11]–[13] are examples of such probing techniques. The differential probe method uses a common server to stores the content of services of interest, and this content is collected from the original servers. The common server thus replays the content and mimics the characteristics that original servers impart in the transfer of their content, like maintaining timing relationships between data packets. In the differential probe, traffic from the replay server must be treated similarly to that treated from the original servers. When it comes to HTTPS traffic, the current tools do not ensure that traffic from replay servers is treated as that originated from an actual server and our work fills this gap. B. Traffic replay mechanisms The traffic replay mechanism mimics the client and serverside behavior for given application data exchange and the underlying protocol. There are many traffic replay tools available. Tcpreplay [22] is one such replay tool that mimics the transport layer behavior for the given stream of transport Fig. 2. Traffic classification of Gmail video play traffic with direct browser accesses and from user-client (TLS handshake not same as browser based access) layer packets. Another example of a layer-specific replay is FlowrReplay that runs at the application layer. The layerspecific replay tools are many times protocol dependent. The technique roleplayer proposed in [23] is capable of replaying application layer data in a protocol-independent manner. The replay layer selection (refer Fig. 1) for traffic replay is crucial as it affects the receiver side’s data collection as well as expected network response. The TCP layer replay adversely affects the traffic analysis as it requires special permission to collect traffic data for analysis. This paper is available on arxiv under CC 4.0 license. Authors: (1) Vinod S. Khandkar and Manjesh K. Hanawal, Industrial Engineering and Operations Research Indian Institute of Technology Bombay, Mumbai, India and {vinod.khandkar, mhanawal}@iitb.ac.in. Authors: Authors: (1) Vinod S. Khandkar and Manjesh K. Hanawal, Industrial Engineering and Operations Research Indian Institute of Technology Bombay, Mumbai, India and {vinod.khandkar, mhanawal}@iitb.ac.in. Table of Links Abstract & Introduction Abstract & Introduction Related Work and Background Related Work and Background Challenges in TD Detection Measurement Setup Development Challenges in TD Detection Measurement Setup Development Case Study : Wehe - TD Detection Tool for Mobile Environment Case Study : Wehe - TD Detection Tool for Mobile Environment Shortcoming of Wehe on HTTPS Traffic Shortcoming of Wehe on HTTPS Traffic TD Detection of HTTPS Traffic TD Detection of HTTPS Traffic Conclusion & References Conclusion & References II. RELATED WORK AND BACKGROUND As net-neutrality regulations were hotly debated in many countries, several authors have proposed methods to detect any net neutrality violations or deliberate traffic discriminations in the Internet. Recent surveys [1], [2] cover all the tools and techniques used by them. The literature is also rich with various forms of discrimination and its detection, like discrimination of content providers [3], end-users [4], specific services like BitTorrent [5]. Our focus in this work is on detecting discrimination of streaming services (both audio and video) that constitute a major portion of internet traffic and consume significant bandwidth. We discuss some of the tools and techniques that is most relevant to our work. A. Existing TD detection tools A. Existing TD detection tools There are two commonly used techniques for detecting TD in users’ Internet traffic. One type of approach passively monitors traffic [6]. In such cases, the end-result or TD result is not immediately available to the user. Instead, the tool provides the aggregated result of traffic differentiation over the given ISP. Another type of detection technique uses specially crafted probing traffic - called Active probes. It analyses network response to probing traffic to detect any anomaly. [6]–[10] describes measurement setups based on such active probing. It uses traffic parameters such as packet loss, latency, packet sequence, or pattern to identify network operation characteristics or detect anomalies. Some tool uses multiple types of probing traffics called active differential probes. While one probing traffic type undergoes standard network middle-box processing, the other probing traffic type is supposed to evade any traffic differentiation by being not amenable to middle-box processing. Typically, in such differential probe methods, one traffic type mimics characters of traffic of services accessed from the original server (like Youtube, Netflix). The other is a reference or control traffic. In this method, network responses to both traffic streams are compared to check if performance of any stream suffered against the other. [11]–[13] are examples of such probing techniques. The differential probe method uses a common server to stores the content of services of interest, and this content is collected from the original servers. The common server thus replays the content and mimics the characteristics that original servers impart in the transfer of their content, like maintaining timing relationships between data packets. In the differential probe, traffic from the replay server must be treated similarly to that treated from the original servers. When it comes to HTTPS traffic, the current tools do not ensure that traffic from replay servers is treated as that originated from an actual server and our work fills this gap. B. Traffic replay mechanisms B. Traffic replay mechanisms The traffic replay mechanism mimics the client and serverside behavior for given application data exchange and the underlying protocol. There are many traffic replay tools available. Tcpreplay [22] is one such replay tool that mimics the transport layer behavior for the given stream of transport Fig. 2. Traffic classification of Gmail video play traffic with direct browser accesses and from user-client (TLS handshake not same as browser based access) layer packets. Another example of a layer-specific replay is FlowrReplay that runs at the application layer. The layerspecific replay tools are many times protocol dependent. The technique roleplayer proposed in [23] is capable of replaying application layer data in a protocol-independent manner. The replay layer selection (refer Fig. 1) for traffic replay is crucial as it affects the receiver side’s data collection as well as expected network response. The TCP layer replay adversely affects the traffic analysis as it requires special permission to collect traffic data for analysis. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Techniques for Detecting Discrimination in Streaming Services

Having the correct data to support your use case is essential to a successful implementation of LLMs in any business. While most out-of-the-box LLMs are great at tasks, they can struggle with specific business problems. They didn’t train on the data for your business problem, so they don’t have adequate context to solve it. general Businesses often have a treasure trove of internal data and documents that could meet this need for specific context. But, here’s the question: How do we integrate all this useful data (context) into the LLM without doing resource-intensive and time-consuming retraining or fine-tuning the LLM? The answer is (RAG), a technique that enhances LLMs with just-in-time retrieval of close context information. retrieval-augmented generation In this post, we’ll walk through how to use and to implement the storage and retrieval of this contextual data for an LLM to use. We’ll solve a context-specific problem with RAG by using LlamaIndex, and then we’ll deploy our solution easily to Heroku. LlamaIndex LangChain Before we start coding, let’s quickly cover the core concepts. A brief introduction to RAG and LlamaIndex When you ask your LLM a question that requires context to answer, RAG retrieves that contextual data to help the LLM give a more accurate and specific response. It’s like having a sous chef quickly run out to the farmer’s market to get the freshest ingredients that weren’t in the pantry so that the executive chef can cook up the perfect dish with all the necessary ingredients. One key to how the RAG workflow can provide context is using vector databases and vector search indexes. Let’s break down some of the core concepts and what this all entails. A is a set of coded numbers that represent the meaning and context of a piece of text (such as a word, phrase, sentence, or even an entire document). vector An is the actual numerical values in a vector; but most people tend to use the terms “vector” and “embedding” interchangeably. embedding An has been trained on documents so that it can convert newly input text into vectors. Not all texts talk about the same things or in the same way — consider a set of academic research papers versus a set of marketing materials. So, we have different embedding models — each trained on specific datasets with specific goals in mind. embedding model Using an embedding model, we can create embeddings from documents, breaking the text in those documents down to their coded numbers. Creating an embedding might involve a strategy like , which splits large documents into smaller, manageable pieces. From there, each chunk is converted into an embedding. document chunking When you query a , your question is turned into an embedding and compared against all of the other embeddings stored in a vector database. When you build a , you can perform very fast and accurate vector searches (also called similarity searches). Using vector databases lets you perform fast and accurate searches — not merely to match the presence of a specific string like in traditional databases, but to match for documents that are similar in with the words you used. vector database vector search index meaning Within the context of RAG, we use our original prompt to perform a vector search against all of the documents in a vector database. Then, those matching documents are sent as context to the LLM application. The LLM now has a set of detailed notes to refer to when crafting its answer to the original prompt. LlamaIndex is a crucial framework that simplifies the process of integrating, organizing, and retrieving private or specialized data. It will help us with creating the document embeddings and vector search index. Then, we’ll lean on LangChain to piece it all together, executing the similarity search and sending the results to our LLM to fetch a response. Together, LlamaIndex and LangChain provide a secure and dependable solution for handling the RAG workflow. Are you ready to build something? Here we go. An introduction to our demo project The best way to learn about RAG with LlamaIndex and Heroku is to build a small sample application. For our purposes, let’s pretend we work with , a library of over 70,000 free ebooks. We want to build an LLM-based chatbot that can answer specific questions about the free books in the project. Project Gutenberg This is the perfect use case for using RAG, and we can use LlamaIndex to take advantage of the large corpus of book texts available to us. To keep our project simple, we’ll use , a popular book from AD 401. The Confessions of St. Augustine The codebase for our completed project can be found in this . If you prefer, you can clone the repo and deploy the application to Heroku. Or, you can follow along step by step to see how we got the code that we did. GitHub repository We’ll follow these general steps: Set up our project. Load the data. Build the index. Store the index. Integrate LangChain. Deploy to Heroku. Step 1: Set up our project Create a new folder for your Python project. Then, activate a and install the initial dependencies that we will need. venv (venv) ~/project$ pip install llama-index langchain langchain-openai Next, we’ll load the data to be indexed. Step 2: Load the data When building an index of internal data for use with RAG, you must gather all your data (text) into a single place. In our example case, that's the text of the book we mentioned above. This is the context that we will soon convert to a vector index of embeddings using LlamaIndex. In the typical use case, your context data will be a large corpus of text that fits the business problem you're trying to solve. For our mini demo, we'll create a subfolder called , and then we'll download the book as a single file in that folder. data (venv) ~/project$ mkdir data

(venv) ~/project$ curl https://www.gutenberg.org/cache/epub/3296/pg3296.txt -o data/confessions.txt

(venv) ~/project$ ls data
confessions.txt Step 3: Build the index After gathering all your data in a single directory, it’s time to build the index. We’ll write a simple Python application that will use LlamaIndex to index the data, and then we’ll query our index. For this to work, you will need an OpenAI account and API key. This is because LlamaIndex uses as the . (Changing those defaults is outside the scope of this article.) OpenAI’s text-embedding-3-small default embedding model In our project root folder, we create a file called . The initial contents look like this: index.py # index.py

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
import os

if os.environ.get('OPENAI_API_KEY') is None:
  exit('You must provide an OPENAI_API_KEY env var.')

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
response = query_engine.query("In which city is Saint Augustine the Bishop?")
print(response) We run our file and receive the expected response: (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
Hippo Of course, we can double-check our data. Looking at the first few lines of the book, we see: THE CONFESSIONS OF SAINT AUGUSTINE

By Saint Augustine

Bishop of Hippo As you can see, LlamaIndex did its job. Our Python application completed our question exactly as we would expect from the vector-indexed data. Step 4: Store the index It’s important to note that, in the above example, we only store our indexed data in memory and not on a disk. Our index — now a series of vector embeddings in memory — will be lost completely after we make our call to the OpenAI model and finish the workflow. Creating a vector index (embeddings) for our text isn’t , so we don’t want to have to recompute those results every time we call the model. It’s best to have a separate workflow where we persist the index to disk. Then, we can reference it at any time later. free A common approach to doing this is to store embeddings in a PostgreSQL database and use to perform similarity search. To keep our demonstration simple, we’ll just store our indexed data as flat files. pgvector So, we add this simple step to our file: index.py PERSIST_DIR='./my_vector_indexes/gutenberg/'
index.storage_context.persist(persist_dir=PERSIST_DIR) Now, after we run our file, we can check for our stored index. (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
Hippo

(venv) ~/project$ tree
.
├── data
│   └── confessions.txt
├── index.py
└── my_vector_indexes
    └── gutenberg
        ├── default__vector_store.json
        ├── docstore.json
        ├── graph_store.json
        ├── image__vector_store.json
        └── index_store.json

3 directories, 7 files Step 5: Integrate LangChain We’ve learned the basics of what a vector index store is and how easy it is to build one. But, to actually build an end-to-end application that chains it all together, we use . This way, we can deploy our solution as an API. Let’s rewrite our index.py code to be a bit more production-worthy. LangChain We’ll show the code below and then explain what we’re doing after. It might look like a lot of code, but we are only adding a few new steps. # index.py

import os
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationalRetrievalChain
from langchain.memory import ConversationBufferWindowMemory
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from langchain_community.retrievers import LlamaIndexRetriever
from fastapi import FastAPI
from pydantic import BaseModel

if os.environ.get('OPENAI_API_KEY') is None:
  exit('You must provide an OPENAI_API_KEY env var.')

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)

# For this demo, we will not persist the index.

retriever = LlamaIndexRetriever(index=index.as_query_engine())

llm = ChatOpenAI(model_name="gpt-3.5-turbo", max_tokens=2048)

memory = ConversationBufferWindowMemory(
  memory_key='chat_history',
  return_messages=True,
  k=3
)

conversation = ConversationalRetrievalChain.from_llm(
  llm=llm, 
  retriever=retriever,
  memory=memory,
  max_tokens_limit=1536  
)

class Prompt(BaseModel):
  question: str

app = FastAPI()

@app.post("/prompt")
async def query_chatbot(prompt: Prompt):
  response = conversation.invoke({'question': prompt.question})
  return response['answer']

if __name__=='__main__':
  import uvicorn
  uvicorn.run(app, host="localhost", port=8000) First, notice that we are now directly using LangChain and OpenAI. We set up our along with some so that our can be “remembered” across subsequent queries. Now, we have an actual ChatBot which we can interact with. llm memory conversation From there, we use to create an API server that listens for requests on the endpoint. Requests to that endpoint are expected to have a request body with a question, which is then passed (along with context from our vector index) to the LLM. FastAPI POST /prompt We use to spin up our server on port 8000. uvicorn Before we can start up our server, let’s add those new Python dependencies: (venv) ~/project$ pip install fastapi pydantic uvicorn Now, it’s time to test. We start by spinning up our server. (venv) ~/project$ OPENAI_API_KEY=sk-******** python index.py
INFO:     Started server process [1101807]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) In another terminal, we send a curl request to our endpoint. $ curl -X POST \
       --header "Content-type:application/json" \
       --data '{"question":"Who is Ambrose?"}' \
       http://localhost:8000/prompt

"Ambrose is a person mentioned in the text provided. He is described as a
respected and celibate man who was esteemed by the author. Ambrose is depicted
as a figure of great honor and excellence, particularly known for his
dedication to reading and studying." Success! Our vector index appears to be up and running, and our chatbot is fully functional. It’s time to deploy. Step 6: Deploy to Heroku After completing the major legwork, we only need to take a few simple steps to deploy our application to Heroku. Create the file with Python dependencies requirements.txt Heroku will need to know what Python dependencies to install when building our project. It looks for this list in a file called . We can easily generate one with this command: requirements.txt (venv) ~/project$ pip freeze > requirements.txt Create the Procfile We also need to tell Heroku how to spin up our Python application. We do this in a file called . Procfile (venv) ~/project$ echo \
   'web: uvicorn index:app --host=0.0.0.0 --port=${PORT}' > Procfile Create the file runtime.txt Lastly, will tell Heroku which Python runtime version we would like to use. runtime.txt (venv) ~/project$ echo 'python-3.11.8' > runtime.txt Those are all the files we need. This is what our project folder structure should look like (we’ve removed the persisted vector index): ~/project$ tree
.
├── data
│   └── confessions.txt
├── index.py
├── Procfile
├── requirements.txt
└── runtime.txt

1 directory, 5 files If you’re working from scratch and didn’t clone the for this demo project, then commit these files to your own Git repository. GitHub repo Create a Heroku app After downloading and installing the , run the following commands. You can choose any name for your app, and you’ll need to provide your unique OpenAI API key. Heroku CLI ~/project$ heroku login

~/project$ heroku apps:create my-llamaindex-app

~/project$ heroku git:remote -a my-llamaindex-app

~/project$ heroku config:add OPENAI_API_KEY=replaceme -a my-llamaindex-app

~/project$ git push heroku main
…
remote: -----> Building on the Heroku-22 stack
remote: -----> Determining which buildpack to use for this app
remote: -----> Python app detected
remote: -----> Using Python version specified in runtime.txt
…
remote: -----> Launching...
remote:        Released v4
remote:        https://my-llamaindex-app-6b48faa3ee6a.herokuapp.com/ deployed to Heroku With our application deployed, we test by sending a curl request to our API server: $ curl -X POST \
       --header "Content-type:application/json" \
       --data '{"question":"Who is Ambrose?"}' \
       https://my-llamaindex-app-6b48faa3ee6a.herokuapp.com/prompt

"Ambrose is a significant figure in the text provided. He is being described
as a respected and happy man, known for his celibacy and his dedication to
reading and studying. He is referred to as a holy oracle and a person of
great influence and wisdom." Keep in mind that the curl call above uses the unique Heroku app URL for our deployment. Yours will be different from what is shown here. We are up and running on Heroku! Conclusion We’ve clearly seen the power of LlamaIndex and the important role it plays when building RAG apps to interact with LLMs. When we can easily add specific data sources as context for an LLM without expensive model retraining — that’s a huge win. It’s a win for companies and developers looking to take their LLM workflows a step further. Combining LlamaIndex with other LangChain toolsets is also seamless and straightforward. Building a chatbot requires only a few lines of additional code. And finally, being able to quickly and easily deploy our solution to Heroku makes our application immediately accessible without any fuss. Simple deployments like this free up developers to focus on the more complicated and important task of building their LLM-based solutions. Happy coding!

Streamlining LLM Implementation: How to Enhance Specific Business Solutions with RAG

The data silos problem is like arthritis for online businesses because almost everyone gets it as they grow old. Businesses interact with customers via websites, mobile apps, H5 pages, and end devices. For one reason or another, it is tricky to integrate the data from all these sources. Data stays where it is and cannot be interrelated for further analysis. That's how data silos come to form. The bigger your business grows, the more diversified customer data sources you will have and the more likely you will be trapped by data silos. This is exactly what happens to the insurance company I'm going to talk about in this post. By 2023, they have already served over 500 million customers and signed 57 billion insurance contracts. When they started to build a customer data platform (CDP) to accommodate such a data size, they used multiple components. Data silos in CDP Like most data platforms, their CDP 1.0 had a batch processing pipeline and a real-time streaming pipeline. Offline data was loaded via Spark jobs to Impala, where it was tagged and divided into groups. Meanwhile, Spark also sent it to NebulaGraph for OneID computation (elaborated later in this post). On the other hand, real-time data was tagged by Flink and then stored in HBase, ready to be queried. That led to a component-heavy computation layer in the CDP: Impala, Spark, NebulaGraph, and HBase. As a result, offline tags, real-time tags, and graph data were scattered across multiple components. Integrating them for further data services was costly due to redundant storage and bulky data transfer. What's more, due to discrepancies in storage, they had to expand the size of the CDH cluster and NebulaGraph cluster, adding to the resource and maintenance costs. Apache Doris-based CDP For CDP 2.0, they decide to introduce a unified solution to clean up the mess. At the computation layer of CDP 2.0, undertakes both real-time and offline data storage and computation. Apache Doris To ingest , they utilize the method. Their 30-thread ingestion test shows that it can perform over 300,000 upserts per second. To load , they use a combination of and Stream Load. In addition, in real-time reporting, where they need to extract data from multiple external data sources, they leverage the feature for . offline data Stream Load real-time data Flink-Doris-Connector Multi-Catalog federated queries The customer analytic workflows on this CDP go like this. First, they sort out customer information; then they attach tags to each customer. Based on the tags, they divide customers into groups for more targeted analysis and operation. Next, I'll delve into these workloads and show you how Apache Doris accelerates them. OneID Has this ever happened to you when you have different user registration systems for your products and services? You might collect the email of UserID A from one product webpage and later the social security number of UserID B from another. Then you find out that UserID A and UserID B actually belong to the same person because they go by the same phone number. That's why OneID arises as an idea. It is to pool the user registration information of all business lines into one large table in Apache Doris, sort it out, and make sure that one user has a unique OneID. This is how they figure out which registration information belongs to the same user, leveraging the functions in Apache Doris. Tagging services This CDP accommodates information of , which come from over and are attached to over in total. 500 million customers 500 source tables 2000 tags By timeliness, the tags can be divided into real-time tags and offline tags. The real-time tags are computed by Apache Flink and written into the flat table in Apache Doris, while the offline tags are computed by Apache Doris as they are derived from the user attribute table, business table, and user behavior table in Doris. Here is the company's best practice in data tagging: 1. Offline tags: During the peaks of data writing, a full update might easily cause an OOM error, given their huge data scale. To avoid that, they utilize the function of Apache Doris and enable . This will cut down memory consumption significantly and maintain system stability during data loading. INSERT INTO SELECT partial column update set enable_unique_key_partial_update=true;
insert into tb_label_result(one_id, labelxx) 
select one_id, label_value as labelxx
from ..... 2. Real-time tags: Partial column updates are also available for real-time tags since real-time tags are updated at different paces. All that is needed is to set to . partial_columns true curl --location-trusted -u root: -H "partial_columns:true" -H "column_separator:," -H "columns:id,balance,last_access_time" -T /tmp/test.csv http://127.0.0.1:48037/api/db1/user_profile/_stream_load 3. High-concurrency point queries: With its current business size, the company is receiving query requests for tags at a concurrency level of over 5000 QPS. They use a combination of strategies to guarantee high performance. Firstly, they adopt a for pre-compilation and pre-execution of SQL. Secondly, they fine-tune the parameters for Doris Backend and the tables to optimize storage and execution. Lastly, they enable as a complement to the column-oriented Apache Doris. Prepared Statement row cache Fine-tune Doris Backend parameters in : be.conf disable_storage_row_cache = false                      
storage_page_cache_limit=40% Fine-tune table parameters upon table creation: enable_unique_key_merge_on_write = true
store_row_column = true
light_schema_change = true 4. Tag computation (join): In practice, many tagging services are implemented by multi-table joins in the database. That often involves more than ten tables. For optimal computation performance, they adopt the strategy in Doris. colocation group Customer Grouping The customer grouping pipeline in CDP 2.0 goes like this: Apache Doris receives SQL from customer service, executes the computation, and sends the result set to S3 object storage via SELECT INTO OUTFILE. The company has divided its customers into 1 million groups. The customer grouping task that used to take to finish now only needs . 50 seconds in Impala 10 seconds in Doris Apart from grouping the customers for more fine-grained analysis, sometimes they do analysis in a reverse direction. That is, to target a certain customer and find out to which groups he/she belongs. This helps analysts understand the characteristics of customers as well as how different customer groups overlap. In Apache Doris, this is implemented by the BITMAP functions: is a fast way to check if a customer is part of a certain group, and , , and are the choices for cross-analysis. BITMAP_CONTAINS BITMAP_OR BITMAP_INTERSECT BITMAP_XOR Conclusion From CDP 1.0 to CDP 2.0, the insurance company adopts Apache Doris, a unified data warehouse, to replace Spark+Impala+HBase+NebulaGraph. That increases their data processing efficiency by breaking down the data silos and streamlining data processing pipelines. In CDP 3.0 to come, they want to group their customer by combining real-time tags and offline tags for more diversified and flexible analysis. The and the team will continue to be a supporting partner during this upgrade. Apache Doris community VeloDB Also published . here

Breaking Down Data Silos: How Apache Doris Streamlines Customer Data Integration

ডাটা সিলোস ব্রেকিং ডাউন: অ্যাপাচি ডরিস কীভাবে গ্রাহক ডেটা ইন্টিগ্রেশনকে স্ট্রীমলাইন করে

Datensilos aufbrechen: Wie Apache Doris die Integration von Kundendaten optimiert

Rompiendo silos de datos: cómo Apache Doris agiliza la integración de datos de clientes

Briser les silos de données : comment Apache Doris rationalise l'intégration des données client

डेटा साइलो को तोड़ना: कैसे अपाचे डोरिस ग्राहक डेटा एकीकरण को सुव्यवस्थित करता है

データサイロの打破: Apache Doris が顧客データ統合を効率化する方法

데이터 사일로 해소: Apache Doris가 고객 데이터 통합을 간소화하는 방법

Quebrando Silos de Dados: Como o Apache Doris Simplifica a Integração de Dados do Cliente

Разрушение хранилищ данных: как Apache Doris оптимизирует интеграцию данных клиентов

Veri Silolarını Parçalamak: Apache Doris Müşteri Veri Entegrasyonunu Nasıl Kolaylaştırır?

Phá vỡ các kho dữ liệu: Cách Apache Doris hợp lý hóa việc tích hợp dữ liệu khách hàng

打破数据孤岛：Apache Doris 如何简化客户数据集成

I blogged about in the past, but I skipped an important method that's worthy of a post of its own: peek. This blog post delves into the practicalities of using to debug Java streams, complete with samples and common pitfalls. Java Stream debugging peek() code Understanding Java Streams Java Streams represent a significant shift in how Java developers work with collections and data processing, introducing a functional approach to handling sequences of elements. Streams facilitate declarative processing of collections, enabling operations such as filter, map, reduce, and more in a fluent style. This not only makes the code more readable but also more concise compared to traditional iterative approaches. A Simple Stream Example To illustrate, consider the task of filtering a list of names to only include those that start with the letter "J" and then transform each name into uppercase. Using the traditional approach, this might involve a loop and some if statements. However, with streams, this can be accomplished in a few lines: List<String> names = Arrays.asList("John", "Jacob", "Edward", "Emily");
// Convert list to stream
List<String> filteredNames = names.stream() 
 // Filter names that start with "J"
 .filter(name -> name.startsWith("J")) 
 // Convert each name to uppercase
 .map(String::toUpperCase) 
 // Collect results into a new list
 .collect(Collectors.toList()); 
System.out.println(filteredNames); Output: [JOHN, JACOB] This example demonstrates the power of Java streams: by chaining operations together, we can achieve complex data transformations and filtering with minimal, readable code. It showcases the declarative nature of streams, where we describe what we want to achieve rather than detailing the steps to get there. What is the Method? peek() At its core, is a method provided by the interface, allowing developers a glance into the elements of a stream without disrupting the flow of its operations. The signature of is as follows: peek() Stream peek() Stream<T> peek(Consumer<? super T> action) It accepts a functional interface, which means it performs an action on each element of the stream without altering them. The most common use case for is logging the elements of a stream to understand the state of data at various points in the stream pipeline. To understand peek, let’s look at a sample similar to the previous one: Consumer peek() List<String> collected = Stream.of("apple", "banana", "cherry")
 .filter(s -> s.startsWith("a"))
 .collect(Collectors.toList());
System.out.println(collected); This code filters a list of strings, keeping only the ones that start with "a.” While it's straightforward, understanding what happens during the filter operation is not visible. Debugging with peek() Now, let's incorporate to gain visibility into the stream: peek() List<String> collected = Stream.of("apple", "banana", "cherry")
 .peek(System.out::println) // Logs all elements
 .filter(s -> s.startsWith("a"))
 .peek(System.out::println) // Logs filtered elements
 .collect(Collectors.toList());
System.out.println(collected); By adding both before and after the operation, we can see which elements are processed and how the filter impacts the stream. This visibility is invaluable for debugging, especially when the logic within the stream operations becomes complex. peek() filter We can't step over stream operations with the debugger, but provides a glance into the code that is normally obscured from us. peek() Uncovering Common Bugs with peek() Filtering Issues Consider a scenario where a filter condition is not working as expected: List<String> collected = Stream.of("apple", "banana", "cherry", "Avocado")
 .filter(s -> s.startsWith("a"))
 .collect(Collectors.toList());
System.out.println(collected); Expected output might be , but let's say we also wanted "Avocado" due to a misunderstanding of the method's behavior. Since "Avocado" is spelled with an upper case "A" this code will return false: . Using , we can observe the elements that pass the filter: ["apple"] startsWith Avocado".startsWith("a") peek() List<String> debugged = Stream.of("apple", "banana", "cherry", "Avocado")
 .peek(System.out::println)
 .filter(s -> s.startsWith("a"))
 .peek(System.out::println)
 .collect(Collectors.toList());
System.out.println(debugged); Large Data Sets In scenarios involving large datasets, directly printing every element in the stream to the console for debugging can quickly become impractical. It can clutter the console and make it hard to spot the relevant information. Instead, we can use in a more sophisticated way to selectively collect and analyze data without causing side effects that could alter the behavior of the stream. peek() Consider a scenario where we're processing a large dataset of transactions, and we want to debug issues related to transactions exceeding a certain threshold: class Transaction {
 private String id;
 private double amount;

 // Constructor, getters, and setters omitted for brevity
}

List<Transaction> transactions = // Imagine a large list of transactions

// A placeholder for debugging information
List<Transaction> highValueTransactions = new ArrayList<>();

List<Transaction> processedTransactions = transactions.stream()
 // Filter transactions above a threshold
 .filter(t -> t.getAmount() > 5000) 
 .peek(t -> {
 if (t.getAmount() > 10000) {
 // Collect only high-value transactions for debugging
 highValueTransactions.add(t);
 }
 })
 .collect(Collectors.toList());

// Now, we can analyze high-value transactions separately, without overloading the console
System.out.println("High-value transactions count: " + 
 highValueTransactions.size()); In this approach, is used to inspect elements within the stream conditionally. High-value transactions that meet a specific criterion (e.g., amount > 10,000) are collected into a separate list for further analysis. This technique allows for targeted debugging without printing every element to the console, thereby avoiding performance degradation and clutter. peek() Addressing Side Effects While streams shouldn't have side effects. In fact, such side effects would break the stream debugger in IntelliJ, which I discussed in the past. It's crucial to note that while collecting data for debugging within avoids cluttering the console, it does introduce a side effect to the stream operation, which goes against the recommended use of streams. Streams are designed to be side-effect-free to ensure predictability and reliability, especially in parallel operations. peek() Therefore, while the above example demonstrates a practical use of for debugging, it's important to use such techniques judiciously. Ideally, this debugging strategy should be temporary and removed once the debugging session is completed to maintain the integrity of the stream's functional paradigm. peek() Limitations and Pitfalls While is undeniably a useful tool for debugging Java streams, it comes with its own set of limitations and pitfalls that developers should be aware of. Understanding these can help avoid common traps and ensure that is used effectively and appropriately. peek() peek() Potential for Misuse in Production Code One of the primary risks associated with is its potential for misuse in production code. Because is intended for debugging purposes, using it to alter state or perform operations that affect the outcome of the stream can lead to unpredictable behavior. This is especially true in parallel stream operations, where the order of element processing is not guaranteed. Misusing in such contexts can introduce hard-to-find bugs and undermine the declarative nature of stream processing. peek() peek() peek() Performance Overhead Another consideration is the performance impact of using . While it might seem innocuous, can introduce a significant overhead, particularly in large or complex streams. This is because every action within is executed for each element in the stream, potentially slowing down the entire pipeline. When used excessively or with complex operations, can degrade performance, making it crucial to use this method judiciously and remove any calls from production code after debugging is complete. peek() peek() peek() peek() peek() Side Effects and Functional Purity As highlighted in the enhanced debugging example, can be used to collect data for debugging purposes, but this introduces side effects to what should ideally be a side-effect-free operation. The functional programming paradigm, which streams are a part of, emphasizes purity and immutability. Operations should not alter state outside their scope. By using to modify external state (even for debugging), you're temporarily stepping away from these principles. While this can be acceptable for short-term debugging, it's important to ensure that such uses of do not find their way into production code, as they can compromise the predictability and reliability of your application. peek() peek() peek() The Right Tool for the Job Finally, it's essential to recognize that is not always the right tool for every debugging scenario. In some cases, other techniques, such as logging within the operations themselves, using breakpoints and inspecting variables in an IDE, or writing unit tests to assert the behavior of stream operations, might be more appropriate and effective. Developers should consider as one tool in a broader debugging toolkit, employing it when it makes sense and opting for other strategies when they offer a clearer or more efficient path to identifying and resolving issues. peek() peek() Navigating the Pitfalls To navigate these pitfalls effectively: Reserve strictly for temporary debugging purposes. If you have a linter as part of your CI tools it might make sense to add a rule that blocks code from invoking . peek() peek() Always remove calls from your code before committing it to your codebase, especially for production deployments. peek() Be mindful of performance implications and the potential introduction of side effects. Consider alternative debugging techniques that might be more suited to your specific needs or the particular issue you're investigating. By understanding and respecting these limitations and pitfalls, developers can leverage to enhance their debugging practices without falling into common traps or inadvertently introducing problems into their codebases. peek() Final Thoughts The method offers a simple yet effective way to gain insights into Java stream operations, making it a valuable tool for debugging complex stream pipelines. By understanding how to use effectively, developers can avoid common pitfalls and ensure their stream operations perform as intended. As with any powerful tool, the key is to use it wisely and in moderation. peek() peek() The true value of is in debugging massive data sets, these elements are very hard to analyze even with dedicated tools. By using we can dig into said data set and understand the source of the issue programmatically. peek() peek() Also published . here

Debugging Java Streams With The Peek Method

Welcome to the digital stage, a dazzling world where you, the streamer, take the spotlight, weaving stories and sharing moments in real-time. Here, speed isn't just a feature; it's the very essence of your performance, the heartbeat of your connection with a global audience. Yet, lurking in the shadows of this dynamic landscape is a notorious villain, a showstopper dreaded by performers and audiences alike: "Buffering." This unwelcome guest introduces annoying pauses during your live streams, holding your viewers in suspense, but for all the wrong reasons. It's the hiccup in your performance you didn't script, the unintended cliffhanger that can turn cheers into jeers. Not quite the dramatic twist you were aiming for, right? Enter the Hero: CDN to the Rescue! So, how do you keep the show going smoothly? The secret weapon against this pesky interruption is something called a Content Delivery Network (CDN). Picture a CDN as your behind-the-scenes crew, strategically positioned worldwide. Their mission? To ensure your live performance reaches the audience super fast, no matter where they're seated in the digital auditorium. How Does This Behind-the-Scenes Crew Work?(CDN) When you go live, rather than your performance trekking from a distant server (which gives buffering a chance to crash your show), it's broadcasted from the CDN team member nearest to your viewers. It's like having a local stage in every city, ensuring your performance is live and direct, without delay. Why This Matters to You, the Performer In a world where the audience's patience wears thinner than a tightrope, even a slight hiccup in your stream can lead to a mass exodus from your digital venue. That's where CDNs shine. They're the unsung heroes keeping the digital stage lit, ensuring your show flows without a hitch and your audience stays captivated. The Secret Sauce: 5centsCDN Among the cadre of digital stagehands, 5centsCDN steals the spotlight. With an extensive network ensuring your performance is as smooth as a rehearsed Broadway number, buffering doesn't stand a chance. Think of as your personal stage manager, guaranteeing a seamless show from curtain rise to curtain call. 5centsCDN Encore! The Standing Ovation Benefits Adopting a CDN isn't just about dodging buffering; it's like unlocking a chest of performance-enhancing treasures: : Like a world tour, a CDN takes your show to viewers far and wide, ensuring no fan misses out, no matter their locale. Global Audience Reach : Keep your audience's eyes glued with crystal-clear streaming, as vivid as live theater, making every detail and emotion pop off the screen. HD Quality : Your performance hits the stage faster than a tap dancer's feet, with nary a pause, keeping the rhythm and energy of your stream alive. Supercharged Speed : Safeguard your content like a backstage pass, keeping digital gatecrashers at bay and ensuring your exclusive content remains exclusive. Fort Knox Security : More show for less dough because an efficient stream doesn't drain your production budget, leaving more resources for creative endeavors. Cost Efficiency Casting Your CDN Sidekick Not all CDNs are ready for the limelight. When scouting for your streaming co-star, consider their global network, ability to dodge internet traffic jams, and, of course, the ticket price. It's like choosing the best supporting actor for your blockbuster show—a crucial decision that can elevate your performance. The Final Bow In the vast theater of the internet, your live stream deserves the main stage. Thanks to CDNs, you can give your audience the seamless, high-definition performance they came for. When it comes to ensuring that no single act is missed, 5centsCDN is the wingman every streamer dreams of. So, here's to uninterrupted streaming, standing ovations, and encores galore! As the curtain falls, remember, in the digital era, every streamer has the potential to be a global sensation. With CDNs like 5centsCDN in your corner, the world is your audience, and every stream is a chance to shine.

No Pause, All Applause: Elevating Your Live Streams with CDN Brilliance

কোন বিরতি নেই, সমস্ত করতালি: CDN ব্রিলিয়ান্সের সাথে আপনার লাইভ স্ট্রিমগুলিকে উন্নত করা৷

Keine Pause, nur Applaus: Verbessern Sie Ihre Live-Streams mit CDN Brilliance

Sin pausas, todo aplausos: eleva tus transmisiones en vivo con CDN Brilliance

Pas de pause, tous des applaudissements : améliorez vos diffusions en direct avec CDN Brilliance

कोई विराम नहीं, पूरी तालियाँ: सीडीएन ब्रिलियंस के साथ अपनी लाइव स्ट्रीम को उन्नत करना

一時停止なし、すべての拍手: CDN の輝きでライブ ストリームを向上

일시 정지 없이 모든 박수 갈채: CDN Brilliance로 라이브 스트림 향상

Sem pausa, todos aplausos: elevando suas transmissões ao vivo com CDN Brilliance

Никаких пауз, одни аплодисменты: улучшите качество своих прямых трансляций с помощью CDN Brilliance

Duraklatma Yok, Alkışlar: Canlı Yayınlarınızı CDN Mükemmelliğiyle Yükseltiyoruz

Không tạm dừng, tất cả đều vỗ tay: Nâng tầm luồng trực tiếp của bạn với CDN rực rỡ

没有停顿，掌声不断：利用 CDN 提升您的直播水平

Building and deploying LLM-based applications has never been easier. Since the launch and wide adoption of ChatGPT near the end of 2022, we’ve seen a storm of news about tools, products, and innovations stemming from large language models (LLMs) and generative AI (GenAI). While many tech fads come and go within a few years, it’s clear that are here to stay. LLMs and GenAI Do you ever wonder about all the tooling going on in the background behind many of these new tools and products? In addition, you might even ask yourself how these tools — leveraged by both developer and end users — are run in production. When you peel back the layers for many of these tools and applications, you’re likely to come across . LangChain, Python, and Heroku These are the pieces that we’re going to play around with in this article. We’ll look at a practical example of how AI/ML developers use them to build and easily deploy complex LLM pipeline components. Demystifying LLM Workflows and Pipelines Machine learning pipelines and workflows can seem like a black box for those new to the AI world. This is even more the case with LLMs and their related tools, as they’re such (relatively) new technologies. Working with LLMs can be challenging, especially as you’re looking to create engineering-hardened and production-ready pipelines, workflows, and deployments. With new tools, rapidly changing documentation, and limited instructions, knowing where to start or what to use can be hard. So, let’s start with the basics of LangChain and Heroku. The tells us this: “ ” documentation for LangChain LangChain is a framework for developing applications powered by language models. Meanwhile, this way: “ ” Heroku describes itself Heroku is a cloud platform that lets companies build, deliver, monitor, and scale apps — we’re the fastest way to go from idea to URL, bypassing all those infrastructure headaches. If we put this in the context of building an LLM application, then LangChain and Heroku are a match made in heaven. We need a well-tested and easy-to-use framework (LangChain) to build our LLM application, and then we need a way to deploy and host that application (Heroku). Let’s look into each of these technologies in more detail. Diving into LangChain Let’s briefly discuss how LangChain is used. LangChain is a framework that assists developers in building applications based on LLM models and use cases. It has support for , , and . For example, let’s say we were building a tool that generates reports based on user input or automates customer support response. LangChain acts as the scaffolding for our project, providing the tools and structure to efficiently integrate language models into our solution. Python JavaScript TypeScript Within LangChain, we have several key components: : The agent is the component that interacts with the language model to perform tasks based on our requirements. This is the brain of our application, using the capabilities of language models to understand and generate text. Agent : These are sequences of actions or processes that our agent follows to accomplish a task. For example, if we were automating customer support, a chain might include accepting a customer query, finding relevant information, and then crafting a response. Chains : Templates provide a way to structure the outputs from the language model. For example, if our application generates reports, then we would leverage a template that helps format these reports consistently based on the model’s output. Templates : This enables developers to up LangChain applications as a REST API. LangServe deploy and serve : This tool helps developers the interactions in their language model applications to get them ready for production. LangSmith evaluate, test, and refine LangChain is a widely adopted framework for building AI and LLM applications, and it’s easy to see why. LangChain provides the functionality to build and deploy products end-to-end. Diving into Heroku Heroku is best known as a cloud platform as a service (PaaS) which makes it incredibly simple to deploy applications to the cloud. Developers often want to focus solely on code and implementation. When you’re already dealing with complex data pipelines and LLM-based applications, you likely don’t have the resources or expertise to deal with infrastructure concerns like servers, networks, and persistent storage. With the ability to easily deploy your apps through Heroku, the major hurdle of production-izing your projects is handled effortlessly. Building with LangChain For a better understanding of how LangChain is used in an LLM application, let’s work through some example problems to make the process clear. In general, we would chain together the following pieces to form a single workflow for an LLM chain: Start with a to generate a prompt based on parameters from the user. prompt template Add a to the chain to retrieve data that the language model was not originally trained on (for example, from a database of documents). retriever Add a to include chat history so that the language model has context for formulating a better response. conversation retrieval chain Add an agent to interact with an actual LLM. LangChain lets us “chain” together the processes that form the base of an LLM application. This makes our implementation easy and approachable. Let’s work with a simple example. In our example, we’ll work with OpenAI. We’ll craft our prompt this way: Tell OpenAI to take on the persona of an encouraging fitness trainer. Input a question from the end user. To keep it nice and simple, we won’t worry about chaining in the retrieval of external data or chat history. Once you get the hang of LangChain, adding other capabilities to your chain is straightforward. On our local machine, we activate a virtual environment. Then, we install the packages we need: (venv) $ pip install langchain langchain_openai We’ll create a new file called . Our basic Python code looks like this: main.py import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm That’s it! In this basic example, we’ve used LangChain to chain together a prompt template and our OpenAI agent. To use this from the command line, we would add the following code: user_input = input("Ask me a question related to your fitness goals.\n")
response = chain.invoke({
  "input": user_input
})

print(response) Let’s test out our application from the command line. (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 
Ask me a question related to your fitness goals.
How do I progress toward holding a plank for 60 seconds?
content="That's a great goal to work towards! To progress towards holding a plank for 
60 seconds, it's important to start with proper form and gradually increase the 
duration of your plank holds. Here are some tips to help you progress:\n\n1. Start 
with shorter durations: Begin by holding a plank for as long as you can with good 
form, even if it's just for a few seconds. Gradually increase the time as you get 
stronger.\n\n2. Focus on proper form: Make sure your body is in a straight line from 
head to heels, engage your core muscles, and keep your shoulders directly over your 
elbows.\n\n3. Practice regularly: Aim to include planks in your workout routine a few 
times a week. Consistency is key to building strength and endurance.\n\n4. Mix it up: 
Try different variations of planks, such as side planks or plank with leg lifts, to 
work different muscle groups and keep your workouts challenging.\n\n5. Listen to your 
body: It's important to push yourself, but also know your limits. If you feel any 
pain or discomfort, stop and rest.\n\nRemember, progress takes time and patience. 
Celebrate each milestone along the way, whether it's holding a plank for a few extra 
seconds or mastering a new plank variation. You've got this!" (I’ve added line breaks above for readability.) That’s a great start. But it would be nice if the output was formatted to be a bit more human-readable. To do that, we simply need to add an to our chain. We’ll use . output parser StrOutputParser import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm | output_parser

user_input = input("Ask me a question related to your fitness goals.\n")
response = chain.invoke({
  "input": user_input
})

print(response) Now, at the command line, our application looks like this: (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 
Ask me a question related to your fitness goals.
How do I learn how to do a pistol squat?
That's a great goal to work towards! Pistol squats can be challenging but with practice and patience, you can definitely learn how to do them.

Here are some steps you can follow to progress towards a pistol squat:

1. Start by improving your lower body strength with exercises like squats, lunges, and step-ups.
2. Work on your balance and stability by practicing single-leg balance exercises.
3. Practice partial pistol squats by lowering yourself down onto a bench or chair until you can eventually perform a full pistol squat.
4. Use a support like a TRX band or a pole to assist you with balance and lowering yourself down until you build enough strength to do it unassisted.

Remember to always warm up before attempting pistol squats and listen to your body to avoid injury. And most importantly, stay positive and patient with yourself as you work towards mastering this challenging exercise. You've got this! The LLM response is now formatted for improved readability. Our chains would be much more complex than this for building powerful LLM applications. But that’s the power and simplicity of LangChain. The framework allows for the modularity of logic specific to your needs so you can easily chain together complex workflows. Now that we have a simple LLM application built, we still need the ability to deploy, host, and serve our application to make it useful. As a developer focused on app building rather than infrastructure, we turn to LangServe and Heroku. Serving with LangServe LangServe helps us interact with a LangChain chain through a REST API. To write the serving portion of a LangChain LLM application, we need three key components: A valid chain ( ) like what we built above An API application framework (such as ) FastAPI Route definitions (just as we would have for building any sort of REST API) The provide some helpful examples of how to get up and running. For our example, we just need to use FastAPI to start up an API server and call from LangServe to make our chain accessible via API endpoints. LangServe docs add_routes() Along with this, we’ll need to make some minor modifications to our existing code. We’ll remove the use of the . This will give callers of our API flexibility in how they want to format and use the output. StrOutputParser We won’t prompt for user input from the command line. The API call request will provide the user’s input. We won’t call because LangServe will make this part of handling the API request. chain.invoke() We make sure to add the FastAPI and LangServe packages to our project: (venv) $ pip install langserve fastapi Our final file looks like this: main.py import os
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from fastapi import FastAPI
from langserve import add_routes

my_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly and encouraging fitness trainer."),
    ("user", "{input}")
])

llm = ChatOpenAI(openai_api_key=os.getenv("OPENAI_API_KEY"))

chain = my_prompt | llm

app = FastAPI(title="Fitness Trainer"")

add_routes(app, chain)

if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="localhost", port=8000) On my local machine (Ubuntu 20.04.6 LTS) running Python 3.8.10, I also needed to install some additional packages to get rid of some warnings. You might not need to do this on your machine. (venv) $ pip install sse_starlette pydantic==1.10.13 Now, we start up our server: (venv) $ OPENAI_API_KEY=insert-key-here python3 main.py 

INFO:     Started server process [629848]
INFO:     Waiting for application startup.

LANGSERVE: Playground for chain "/" is live at:
LANGSERVE:  │
LANGSERVE:  └──> /playground/
LANGSERVE:
LANGSERVE: See all available routes at /docs/

INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) Ooooh… nice! In the browser, we can go to . This is what we see: http://localhost:8000/docs LangServe serves up an API docs page that uses a Swagger UI! These are the endpoints now available to us through LangServe. We send a request to the endpoint. But LangServe also gives us a endpoint with a web interface to work with our chain directly. could POST invoke/ playground/ We provide input and click . Here’s the result: Start It’s important to stress the importance of having APIs in the context of LLM application workflows. If you think about it, most use cases of LLMs and applications built on top of them can’t rely on models and resources for inference. This neither makes sense nor scales well. local The real power of LLM applications is the ability to abstract away the complex workflow we’ve described so far. We want to put everything we’ve done behind an API so the use case can scale and others can integrate it. This is only possible if we have an easy option to host and serve these APIs. And that’s where Heroku comes in. Deploying to Heroku Heroku is the key, final part of our LLM application implementation. We have LangChain piece together our workflow, and LangServe will serve it up as a useful REST API. Now, instead of manually setting up complex resources to host and serve traffic, we turn to Heroku for the simple deployment of our application. After , we’re nearly ready to deploy. Let’s walk through the steps. setting up a Heroku account Create a new Heroku app Using the Heroku CLI, we log in and create a new app. $ heroku login
$ heroku create my-langchain-app Set config variables Next, we need to set the environment variable in our Heroku app environment. OPENAI_API_KEY $ heroku config:set OPENAI_API_KEY=replace-with-your-openai-api-key Create config files for Python application deployment To let Heroku know what we need for our Python application to run, we need to create three simple files: : Declares what command Heroku should execute to start our app. Procfile : Specifies the Python package dependencies that Heroku will need to install. requirements.txt : Specifies the exact version of the Python runtime we want to use for our app. runtime.txt These files are quick and easy to create. Each one goes into the project’s root folder. To create the , we run this command: Procfile $ echo 'web: uvicorn main:app --host=0.0.0.0 --port=${PORT}' > Procfile This tells Heroku to run , which is a web server implementation in Python. uvicorn For , we can use the command to output the list of installed packages. requirements.txt pip freeze $ pip freeze > requirements.txt Lastly, for , we will use Python 3.11.8. runtime.txt $ echo 'python-3.11.8' > runtime.txt With these files in place, our project root folder should look like this: $ tree
.
├── main.py
├── Procfile
├── requirements.txt
└── runtime.txt

0 directories, 4 files We commit all of these files to the GitHub repository. Connect Heroku to GitHub repo The last thing to do is for our GitHub repo and then push our code to the remote. Heroku will detect the push of new code and then deploy that code to our application. create a Heroku remote $ heroku git:remote -a my-langchain-app
$ git push heroku main When our code is pushed to the Heroku remote, Heroku builds the application, installs dependencies, and then runs the command in our . The final result of our command looks like this: Procfile git push …

remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:        Done: 71.8M
remote: -----> Launching...
remote:        Released v4
remote:        https://my-langchain-app-ea95419b2750.herokuapp.com/ deployed to Heroku
remote: 
remote: Verifying deploy... done. This shows the URL for our Heroku app. In our browser, we visit . https://my-langchain-app-ea95419b2750.herokuapp.com/playground We also check out our Swagger UI docs page at . https://my-langchain-app-ea95419b2750.herokuapp.com/docs And just like that, we’re up and running! This process is the best way to reduce developer time and overhead when working on large, complex LLM pipelines with LangChain. The ability to take APIs built with LangChain and seamlessly deploy to Heroku with a few simple command line arguments is what makes the pairing of LangChain and Heroku a no-brainer. Conclusion Businesses and developers today are right to ride the wave of AI and LLMs. There’s so much room for innovation and new development in these areas. However, the difference between the successes and failures will depend a lot on the toolchain they use to build and deploy these applications. Using the LangChain framework makes the process of building LLM-based applications approachable and repeatable. But, implementation is only half the battle. Once your application is built, you need the ability to easily and quickly deploy those application APIs into the cloud. That’s where you’ll have the advantage of faster iteration and development, and Heroku is a great way to get you there. Also published . here

Streamlining LLM Application Development and Deployment with LangChain, Heroku, and Python

We all remember the good old days of waiting the whole day for Scooby Doo telecast on Cartoon Network, The Late Late Toy Show on RTE and Hannah Montana on Disney Channel. The television sets, though bulky, promised us endless hours of entertainment. The excitement of flipping through channels, waiting for my favorite show to air, and the family gatherings around the TV for movie nights are etched in my memory. But as we stand at the crossroads of technological evolution, I'm torn between the emotional attachment to cable TV and the undeniable convenience of streaming services. This internal debate leads me to ponder: in our rush to embrace the new, are we too quick to dismiss the value of the old? The Magic of Old-School TV There's an indescribable nostalgia tied to television sets. Scheduled programming meant that you had to plan your day around your favorite show, creating a sense of anticipation that's somewhat lost in today's on-demand culture. Channel surfing led to unexpected discoveries and by chance stumbling upon shows or movies that would become new favorites. Often, I recall the joy of finding a classic film on a lazy Sunday afternoon or the entire family gathering to watch a live broadcast of a major event, moments that felt like shared adventures. Cable TV was simple and reliable. There were no buffering issues, no need for a strong internet connection, just instant access to entertainment. In contrast, streaming can sometimes feel like a chore—deciding what to watch from thousands of options can be overwhelming, and technical glitches can disrupt the streaming experience. The Rise of Streaming Services Despite my fondness for cable, the allure of streaming services is undeniable. The convenience of watching anything at any time, without commercial interruptions, is a game-changer. My journey of adapting to streaming was met with resistance at first. The idea of letting go of the familiarity of TV for a digital platform felt like betraying a lifelong friend. Yet, I couldn't ignore the benefits. Streaming services have introduced me to shows and movies from around the world, broadening my horizons and becoming part of my personal and cultural lexicon. The statistics speak volumes about the streaming revolution. As of 2023, over to at least one streaming service, with the average subscriber spending more on streaming than on traditional cable subscriptions. This shift highlights the growing preference for the flexibility and variety that streaming offers. 80% of U.S. households have a subscription The Emotional Tug-of-War The transition from cable to streaming hasn't been without its emotional conflicts. There are moments when I miss the simplicity of cable TV, the ritual of sitting down at a specific time to watch a show, and the element of surprise that came with channel surfing. It's akin to saying goodbye to an old friend, a staple of my daily life that's slowly fading into the background. Yet, I recognize the practicality of streaming services. The ability to pause, rewind, or binge-watch a series fits seamlessly into the hectic pace of modern life. It's a convenience that, once experienced, is hard to give up. The Social Aspect One of the most significant changes brought about by streaming is how it has altered our viewing habits. Cable TV was a communal experience, a reason for family and friends to come together. Streaming, on the other hand, tends to be more individualized, with each person diving into their personal preferences, often in isolation. I've made efforts to recreate the communal aspect of old-school TV with streaming services, organizing virtual watch parties or sharing recommendations with friends and family. While it's not quite the same, it's a testament to how we're adapting our social interactions around the new ways we consume media. Can We Find a Middle Ground? As I reflect on my journey from cable TV to streaming, I wonder if it's really necessary to choose one over the other. Perhaps there's a way to balance the nostalgia for cable with the advantages of streaming. I've found joy in dedicating specific nights to movie watching, akin to the scheduled programming of old, and exploring classic shows available on streaming platforms. Conclusion The shift from cable TV to streaming services is more than just a change in how we access entertainment; it's a reflection of our evolving relationship with technology. While I cherish the memories and simplicity of television programming, I also appreciate the innovation and convenience that streaming brings to our lives. It's not about saying goodbye to the past but learning to blend the old with the new, creating new memories and experiences in this digital age. As we navigate these changes, it's comforting to know that at the heart of it all, it's the stories we watch and the people we share them with that truly matter.

Stories, News & Mentions about OSTREAM

Ostream’s Stories on HackerNoon