Hollywood, Health & Society (HH&S), a program of the USC Annenberg Norman Lear Center, has unveiled the winners of the 2024 Sentinel Awards, which recognize some of the past year’s best TV entertainment that responsibly informs audiences on critical topics.

"Abbott Elementary," "Expats" and "Hacks" Are Among Shows Set To Receive Sentinel Awards - SHOOTonline

USC Annenberg Norman Lear Center's Hollywood, Health & Society program will honor advocacy stories on 'Grey's Anatomy' and more with Sentinel Awards.

‘Grey’s Anatomy,’ ‘Hacks,’ ‘The Simpsons’ Among Series to Be Honored by Hollywood, Health & Society’s Sentinel Awards

The 2024 Sentinel Awards, organized by the Hollywood, Health & Society program at the USC Annenberg Norman Lear Center, will honor shows like 'Hacks.'

Sentinel Awards 2024 Winners Include 'The Morning Show,' 'Hacks,' More

Laurens County will see nearly 500 new jobs come to the area after an international automotive parts company opens their new manufacturing facility.

More than 450 jobs are coming to Laurens County as auto parts company opens facility

Real-time index price for S&P 500 Growth (IGX), along with buy or sell indicators, analysis, charts, historical performance, news and more

Consumer Discretionary Stocks Are at It Again! Don’t Miss Out on These Potential Winners!

The United Auto Workers union says it reached a tentative contract with General Motors, the last of the Detroit Three automakers to agree to a deal. Under the deal reached

UAW reaches deal with General Motors that ends strikes against Detroit automakers pending votes

The supplier of seats for the Ford Bronco has given pay raises to workers at its plant in Detroit in a bid to keep them from leaving for other jobs and to protect its business as it ramps up for the post-strike production rush.

UAW strike spurs pay raise for workers at Ford supplier

Fintel reports that on October 30, 2023, Barclays maintained coverage of Lear (NYSE:LEA) with a Overweight recommendation. Analyst Price Forecast Suggests 36.07% Upside As of October 5, 2023, the average one-year price target for Lear is 174.

Barclays Maintains Lear (LEA) Overweight Recommendation

Fintel reports that on October 30, 2023, Benchmark reiterated coverage of Lear (NYSE:LEA) with a Buy recommendation. Analyst Price Forecast Suggests 36.07% Upside As of October 5, 2023, the average one-year price target for Lear is 174.

Benchmark Reiterates Lear (LEA) Buy Recommendation

Automotive tier-1 supplier Lear Corporation has reported its results for the third quarter of 2023, with a sales increase of 10% year-on-year to US$5.8 billion.

Fifth consecutive quarter of growth for Lear Corp in Q3

Lear

lear

Table of Links Abstract and Introduction


Domain and Task
2.1. Data sources and complexity
2.2. Task definition


Related Work
3.1. Text mining and NLP research overview
3.2. Text mining and NLP in industry use
3.3. Text mining and NLP for procurement
3.4. Conclusion from literature review


Proposed Methodology
4.1. Domain knowledge
4.2. Content extraction
4.3. Lot zoning
4.4. Lot item detection
4.5. Lot parsing
4.6. XML parsing, data joining, and risk indices development


Experiment and Demonstration
5.1. Component evaluation
5.2. System demonstration


Discussion
6.1. The ‘industry’ focus of the project
6.2. Data heterogeneity, multilingual and multi-task nature
6.3. The dilemma of algorithmic choices
6.4. The cost of training data


Conclusion, Acknowledgements, and References 5. Experiment and Demonstration In this section, we present evaluation of some of the components above and demonstrate part of the end system. 5.1. Component evaluation Here, we present evaluations of lot zoning and lot item detection, for which our domain experts were able to create some gold standard for model building and evaluation. In both tasks, we use text classification and compare three machine learning algorithms: linear SVM, logistic regression (LR) and random forest (RF). Therefore, we use the standard evaluation metrics for classification: Precision, Recall and F1. These are calculated as follows. Since each task has essentially two classes, we measure the scores for each class and take their average for reporting (macro-average). To evaluate lot zoning at page and table levels, we created two separate datasets. We asked domain experts from Vamstar to annotate a random collection of pages and tables extracted from our raw TED datasets into ‘relevant’ or ‘irrelevant’. This gave us a total of 781 pages, and 515 tables (multilingual in both cases). To evaluate lot item detection, we conducted two sets of experiments. First, using the TED database created by Vamstar, we identify the field containing values most similar to the description of lots and items, i.e., ‘lot and item descriptions’, split the values based on sentence boundaries and take the unique values. Values with less than 2 words or more than 20 words are excluded. This gives us a total of 18363 positive examples (multilingual). For negative examples, we apply the same process to the other four fields: ‘name and addresses’ of buyers, ‘notice title’, ‘short description’, and ‘contract criteria’. This gives us a total of 15632 negative examples. Combined together, we refer to this dataset as ‘lot item gold standard’. However, this only allows evaluating sentence-level classifiers, and the dataset does not necessarily represent the real documents that may describe lots in free-text and tables. Therefore, our second evaluation involves domain experts manually verifying the extractions (sentences, table rows) of the best model from 20 English tender notices for precision only. We refer to this dataset as ‘lot item verification’. Statistics of these datasets are shown in Table 1. Tables 2 ~ 3 show the evaluation results for different tasks respectively. The results show that arguably, random forest is the most robust algorithm as it performed best on page-level lot zoning and lot item detection, while achieving comparable results on table-level lot zoning. We also conducted error analysis to understand where the automated methods failed. In terms of the two ‘zoning’ tasks, we found three main causes that applied to both pages and tables. First, the low recall is typically due to ‘sparse’signals, such as a tender containing only one lot with very few items, described in a very short table or paragraph/list. Content as such is difficult to extract because the features generated for those text passages would be sparse compared to ‘denser’ pages or larger tables. Low precisions are primarily due to ‘noise’ in the domain lexicon, as some generic words (e.g. ‘solution’, ‘wrap’, ‘free’ as in ‘sugar free’) were retained and scored high. To address these issues, domain experts suggested developing ‘post processing’ rules to ‘recover’ passages that are potentially valid candidates for processing. For example, enforcing that at least one text passage must be extracted from each tender (thus those not making the classifier decisions may be recovered). Also, excluding unigrams in the domain lexicon could help address false positives. The third common cause of errors is due to content extraction from PDFs. As an example, arbitrary whitespaces are recognised between every letter when parsing some PDFs (e.g., ‘book’ becomes ‘b o o k’). This is an issue that is much harder to address, as it depends on the quality of PDFs as well as the third party extraction tool. This is an example of the practical challenges that industrial text mining and NLP projects can face while research may not as they deal with much higher quality data. This implies that in reality, system performance may be limited by some obstacles that are hard to overcome. In terms of the lot item detection task, when examining the ‘lot item verification’ data, we find that the most common issue causing the decrease in precision is ‘noisy’ domain lexicon entries, as explained above. Further, complexity in the domain vocabulary also made the task challenging. For example, pharmaceutical ingredients sometimes have long names and are written in certain patterns, such as ‘17-alphahydroxyprogesterone’. For cases like these, domain experts suggested developing certain ‘regular expressions’ incorporating domain dictionaries. This implies that industrial text mining and NLP projects are often more complicated than tweaking a machine learning based method for performance measured by established standards. Practical solutions often rely on incorporating domain knowledge in various forms, oftentimes rules. This is consistent with earlier findings such as Chiticariu et al. (2013). Authors:
(1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk);
(2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io);
(3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io);
(4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk);
(5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). This paper is available on arxiv under CC BY 4.0 license. Table of Links Abstract and Introduction Domain and Task
2.1. Data sources and complexity
2.2. Task definition Related Work
3.1. Text mining and NLP research overview
3.2. Text mining and NLP in industry use
3.3. Text mining and NLP for procurement
3.4. Conclusion from literature review Proposed Methodology
4.1. Domain knowledge
4.2. Content extraction
4.3. Lot zoning
4.4. Lot item detection
4.5. Lot parsing
4.6. XML parsing, data joining, and risk indices development Experiment and Demonstration
5.1. Component evaluation
5.2. System demonstration Discussion
6.1. The ‘industry’ focus of the project
6.2. Data heterogeneity, multilingual and multi-task nature
6.3. The dilemma of algorithmic choices
6.4. The cost of training data Conclusion, Acknowledgements, and References Abstract and Introduction Abstract and Introduction Abstract and Introduction Abstract and Introduction Domain and Task 2.1. Data sources and complexity 2.2. Task definition Domain and Task Domain and Task 2.1. Data sources and complexity 2.1. Data sources and complexity 2.2. Task definition 2.2. Task definition Related Work 3.1. Text mining and NLP research overview 3.2. Text mining and NLP in industry use 3.3. Text mining and NLP for procurement 3.4. Conclusion from literature review Related Work Related Work 3.1. Text mining and NLP research overview 3.1. Text mining and NLP research overview 3.2. Text mining and NLP in industry use 3.2. Text mining and NLP in industry use 3.3. Text mining and NLP for procurement 3.3. Text mining and NLP for procurement 3.4. Conclusion from literature review 3.4. Conclusion from literature review Proposed Methodology 4.1. Domain knowledge 4.2. Content extraction 4.3. Lot zoning 4.4. Lot item detection 4.5. Lot parsing 4.6. XML parsing, data joining, and risk indices development Proposed Methodology Proposed Methodology Proposed Methodology 4.1. Domain knowledge 4.1. Domain knowledge 4.2. Content extraction 4.2. Content extraction 4.3. Lot zoning 4.3. Lot zoning 4.4. Lot item detection 4.4. Lot item detection 4.5. Lot parsing 4.5. Lot parsing 4.6. XML parsing, data joining, and risk indices development 4.6. XML parsing, data joining, and risk indices development Experiment and Demonstration 5.1. Component evaluation 5.2. System demonstration Experiment and Demonstration Experiment and Demonstration 5.1. Component evaluation 5.1. Component evaluation 5.2. System demonstration 5.2. System demonstration Discussion 6.1. The ‘industry’ focus of the project 6.2. Data heterogeneity, multilingual and multi-task nature 6.3. The dilemma of algorithmic choices 6.4. The cost of training data Discussion Discussion 6.1. The ‘industry’ focus of the project 6.1. The ‘industry’ focus of the project 6.2. Data heterogeneity, multilingual and multi-task nature 6.2. Data heterogeneity, multilingual and multi-task nature 6.3. The dilemma of algorithmic choices 6.3. The dilemma of algorithmic choices 6.4. The cost of training data 6.4. The cost of training data Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References 5. Experiment and Demonstration In this section, we present evaluation of some of the components above and demonstrate part of the end system. 5.1. Component evaluation Here, we present evaluations of lot zoning and lot item detection, for which our domain experts were able to create some gold standard for model building and evaluation. In both tasks, we use text classification and compare three machine learning algorithms: linear SVM, logistic regression (LR) and random forest (RF). Therefore, we use the standard evaluation metrics for classification: Precision, Recall and F1. These are calculated as follows. Since each task has essentially two classes, we measure the scores for each class and take their average for reporting (macro-average). To evaluate lot zoning at page and table levels, we created two separate datasets. We asked domain experts from Vamstar to annotate a random collection of pages and tables extracted from our raw TED datasets into ‘relevant’ or ‘irrelevant’. This gave us a total of 781 pages, and 515 tables (multilingual in both cases). To evaluate lot item detection, we conducted two sets of experiments. First, using the TED database created by Vamstar, we identify the field containing values most similar to the description of lots and items, i.e., ‘lot and item descriptions’, split the values based on sentence boundaries and take the unique values. Values with less than 2 words or more than 20 words are excluded. This gives us a total of 18363 positive examples (multilingual). For negative examples, we apply the same process to the other four fields: ‘name and addresses’ of buyers, ‘notice title’, ‘short description’, and ‘contract criteria’. This gives us a total of 15632 negative examples. Combined together, we refer to this dataset as ‘lot item gold standard’. However, this only allows evaluating sentence-level classifiers, and the dataset does not necessarily represent the real documents that may describe lots in free-text and tables. Therefore, our second evaluation involves domain experts manually verifying the extractions (sentences, table rows) of the best model from 20 English tender notices for precision only. We refer to this dataset as ‘lot item verification’. Statistics of these datasets are shown in Table 1. Tables 2 ~ 3 show the evaluation results for different tasks respectively. The results show that arguably, random forest is the most robust algorithm as it performed best on page-level lot zoning and lot item detection, while achieving comparable results on table-level lot zoning. We also conducted error analysis to understand where the automated methods failed. In terms of the two ‘zoning’ tasks, we found three main causes that applied to both pages and tables. First, the low recall is typically due to ‘sparse’signals, such as a tender containing only one lot with very few items, described in a very short table or paragraph/list. Content as such is difficult to extract because the features generated for those text passages would be sparse compared to ‘denser’ pages or larger tables. Low precisions are primarily due to ‘noise’ in the domain lexicon, as some generic words (e.g. ‘solution’, ‘wrap’, ‘free’ as in ‘sugar free’) were retained and scored high. To address these issues, domain experts suggested developing ‘post processing’ rules to ‘recover’ passages that are potentially valid candidates for processing. For example, enforcing that at least one text passage must be extracted from each tender (thus those not making the classifier decisions may be recovered). Also, excluding unigrams in the domain lexicon could help address false positives. The third common cause of errors is due to content extraction from PDFs. As an example, arbitrary whitespaces are recognised between every letter when parsing some PDFs (e.g., ‘book’ becomes ‘b o o k’). This is an issue that is much harder to address, as it depends on the quality of PDFs as well as the third party extraction tool. This is an example of the practical challenges that industrial text mining and NLP projects can face while research may not as they deal with much higher quality data. This implies that in reality, system performance may be limited by some obstacles that are hard to overcome. In terms of the lot item detection task, when examining the ‘lot item verification’ data, we find that the most common issue causing the decrease in precision is ‘noisy’ domain lexicon entries, as explained above. Further, complexity in the domain vocabulary also made the task challenging. For example, pharmaceutical ingredients sometimes have long names and are written in certain patterns, such as ‘17-alphahydroxyprogesterone’. For cases like these, domain experts suggested developing certain ‘regular expressions’ incorporating domain dictionaries. This implies that industrial text mining and NLP projects are often more complicated than tweaking a machine learning based method for performance measured by established standards. Practical solutions often rely on incorporating domain knowledge in various forms, oftentimes rules. This is consistent with earlier findings such as Chiticariu et al. (2013). Authors: (1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk); (2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io); (3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io); (4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk); (5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). Authors: Authors: (1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk); (2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io); (3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io); (4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk); (5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). This paper is available on arxiv under CC BY 4.0 license. This paper is available on arxiv under CC BY 4.0 license. available on arxiv available on arxiv

How Machine Learning Helps Identify Lot Items in Tender Documents

Table of Links Abstract and 1. Introduction 2. Preliminaries and 2.1. Blind deconvolution 2.2. Quadratic neural networks 3. Methodology 3.1. Time domain quadratic convolutional filter 3.2. Superiority of cyclic features extraction by QCNN 3.3. Frequency domain linear filter with envelope spectrum objective function 3.4. Integral optimization with uncertainty-aware weighing scheme 4. Computational experiments 4.1. Experimental configurations 4.2. Case study 1: PU dataset 4.3. Case study 2: JNU dataset 4.4. Case study 3: HIT dataset 5. Computational experiments 5.1. Comparison of BD methods 5.2. Classification results on various noise conditions 5.3. Employing ClassBD to deep learning classifiers 5.4. Employing ClassBD to machine learning classifiers 5.5. Feature extraction ability of quadratic and conventional networks 5.6. Comparison of ClassBD filters 6. Conclusions Appendix and References 5.3. Employing ClassBD to deep learning classifiers Given that ClassBD serves as a signal preprocessing module, it possesses the flexibility to be integrated into various backbone networks. In the context of this experiment, we assess the classification performance by incorporating ClassBD into four widely recognized deep learning classifiers: ResNet [85], MobileNetV3 [86], WDCNN [54], and Transformer [87]. It is important to note that some networks were initially designed for image classification, so some parameters are revised to accommodate the 1D signal input. The properties of these backbone networks are illustrated in Table 11. Firstly, the classification results on the PU dataset are illustrated in Table 12. Predominantly, ClassBD has demonstrated its efficacy in enhancing performance. For instance, in the case of N09M07F10 with -4dB noise, the application of ClassBD results in an improvement exceeding 10% across all backbones. On average, when the SNR is -4dB, ClassBD achieves F1 scores of 87.60%, whereas the raw networks only yield 64.07%. Subsequently, we also test these models to the JNU and HIT datasets, setting the SNRs to -6dB and -4dB respectively. As depicted in Table 13, ClassBD exhibits commendable performance on both datasets, and all backbones experience a performance boost. Specifically, on the JNU dataset, the F1 scores exceed 90% post the employment of ClassBD, irrespective of the backbone performance. Furthermore, a substantial improvement is observed in the Transformer. The raw Transformer initially yields F1 scores around 50% on the JNU and HIT datasets, which, after the application of ClassBD, escalates to an average F1 score of 90%. Finally, the results on the three datasets suggest that ClassBD can function as a plug-and-play denoising module, thereby enhancing the performance of deep learning classifiers under substantial noise. Considering both performance and model size, the simplest backbone, WDCNN, achieves consistent performance under all conditions. Consequently, we recommend it as the backbone for bearing fault diagnosis. Authors:
(1) Jing-Xiao Liao, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China and School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China;
(2) Chao He, School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, China;
(3) Jipu Li, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China;
(4) Jinwei Sun, School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China;
(5) Shiping Zhang (Corresponding author), School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China;
(6) Xiaoge Zhang (Corresponding author), Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2. Preliminaries and 2.1. Blind deconvolution 2. Preliminaries and 2.1. Blind deconvolution 2.2. Quadratic neural networks 2.2. Quadratic neural networks 3. Methodology 3. Methodology 3.1. Time domain quadratic convolutional filter 3.1. Time domain quadratic convolutional filter 3.2. Superiority of cyclic features extraction by QCNN 3.2. Superiority of cyclic features extraction by QCNN 3.3. Frequency domain linear filter with envelope spectrum objective function 3.3. Frequency domain linear filter with envelope spectrum objective function 3.4. Integral optimization with uncertainty-aware weighing scheme 3.4. Integral optimization with uncertainty-aware weighing scheme 4. Computational experiments 4.1. Experimental configurations 4.1. Experimental configurations 4.2. Case study 1: PU dataset 4.2. Case study 1: PU dataset 4.3. Case study 2: JNU dataset 4.3. Case study 2: JNU dataset 4.4. Case study 3: HIT dataset 4.4. Case study 3: HIT dataset 5. Computational experiments 5.1. Comparison of BD methods 5.1. Comparison of BD methods 5.2. Classification results on various noise conditions 5.2. Classification results on various noise conditions 5.3. Employing ClassBD to deep learning classifiers 5.3. Employing ClassBD to deep learning classifiers 5.4. Employing ClassBD to machine learning classifiers 5.4. Employing ClassBD to machine learning classifiers 5.5. Feature extraction ability of quadratic and conventional networks 5.5. Feature extraction ability of quadratic and conventional networks 5.6. Comparison of ClassBD filters 5.6. Comparison of ClassBD filters 6. Conclusions 6. Conclusions Appendix and References Appendix and References 5.3. Employing ClassBD to deep learning classifiers Given that ClassBD serves as a signal preprocessing module, it possesses the flexibility to be integrated into various backbone networks. In the context of this experiment, we assess the classification performance by incorporating ClassBD into four widely recognized deep learning classifiers: ResNet [85], MobileNetV3 [86], WDCNN [54], and Transformer [87]. It is important to note that some networks were initially designed for image classification, so some parameters are revised to accommodate the 1D signal input. The properties of these backbone networks are illustrated in Table 11. Firstly, the classification results on the PU dataset are illustrated in Table 12. Predominantly, ClassBD has demonstrated its efficacy in enhancing performance. For instance, in the case of N09M07F10 with -4dB noise, the application of ClassBD results in an improvement exceeding 10% across all backbones. On average, when the SNR is -4dB, ClassBD achieves F1 scores of 87.60%, whereas the raw networks only yield 64.07%. Subsequently, we also test these models to the JNU and HIT datasets, setting the SNRs to -6dB and -4dB respectively. As depicted in Table 13, ClassBD exhibits commendable performance on both datasets, and all backbones experience a performance boost. Specifically, on the JNU dataset, the F1 scores exceed 90% post the employment of ClassBD, irrespective of the backbone performance. Furthermore, a substantial improvement is observed in the Transformer. The raw Transformer initially yields F1 scores around 50% on the JNU and HIT datasets, which, after the application of ClassBD, escalates to an average F1 score of 90%. Finally, the results on the three datasets suggest that ClassBD can function as a plug-and-play denoising module, thereby enhancing the performance of deep learning classifiers under substantial noise. Considering both performance and model size, the simplest backbone, WDCNN, achieves consistent performance under all conditions. Consequently, we recommend it as the backbone for bearing fault diagnosis. Authors: (1) Jing-Xiao Liao, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China and School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (2) Chao He, School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, China; (3) Jipu Li, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China; (4) Jinwei Sun, School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (5) Shiping Zhang (Corresponding author), School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (6) Xiaoge Zhang (Corresponding author), Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China. Authors: Authors: (1) Jing-Xiao Liao, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China and School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (2) Chao He, School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, China; (3) Jipu Li, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China; (4) Jinwei Sun, School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (5) Shiping Zhang (Corresponding author), School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin, China; (6) Xiaoge Zhang (Corresponding author), Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, Special Administrative Region of China. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv

ClassBD Boosts Deep Learning Classifiers for Better Fault Diagnosis

Table of Links Abstract and Introduction


Domain and Task
2.1. Data sources and complexity
2.2. Task definition


Related Work
3.1. Text mining and NLP research overview
3.2. Text mining and NLP in industry use
3.3. Text mining and NLP for procurement
3.4. Conclusion from literature review


Proposed Methodology
4.1. Domain knowledge
4.2. Content extraction
4.3. Lot zoning
4.4. Lot item detection
4.5. Lot parsing
4.6. XML parsing, data joining, and risk indices development


Experiment and Demonstration
5.1. Component evaluation
5.2. System demonstration


Discussion
6.1. The ‘industry’ focus of the project
6.2. Data heterogeneity, multilingual and multi-task nature
6.3. The dilemma of algorithmic choices
6.4. The cost of training data


Conclusion, Acknowledgements, and References 4.4. Lot item detection With the relevant pages and tables identified, our next step in the process is to identify the text elements in the pages/tables that actually describe lots and items. We would like to capture two kinds of information: the texts that indicate a lot reference and the texts that describe individual items in a lot. In practice, this is often contained in a single, coherent text passage, such as that shown in Figure 5. Thus we handle this as a binary sentence classification task where a sentence is a positive example if it contains either a lot reference, or item information, or both; and negative otherwise. We deter the extraction of lot reference and structured item information to the next component ‘lot parsing’. We also need to deal with unstructured texts from a page and content from tables in slightly different ways. 4.4.1. Unstructured texts in a page For unstructured texts in a page, we apply sentence splitting and then sentence classification. For features, we adapt the 24 page-level features explained above in the ‘page selection’section by applying them to each sentence to classify. As an example, in equation 1, we replace 𝑝 with the input sentence 𝑠 and by alternating the 𝑚𝑒𝑡𝑟𝑖𝑐 we obtain 4 features. The same can be applied to equations 2~3. Then replacing 𝐿 with the other two domain dictionaries will produce another 8 features. Further, we add three binary features that help capture lot references: ● If the sentence contains the word ‘lot’ (and its translation in other languages) ● If the sentence contains the word ‘number’ or ‘no.’ or ‘num’ (and its translations) ● If the sentence contains number-like tokens (including roman/arabic numerals and patterns such as ‘1.1, 1.21, X.2, II.1’) We also compare the same set of algorithms for classification. 4.4.2. Tables Similar to the idea explained in passage selection, we simply treat each row as a sentence and therefore, the solution would be almost the same as that for dealing with unstructured texts above. One exception is for tables like that in Figure 2, where the lot references need to be interpreted based on combining the table header ‘Lot Number’ and the text in the row spans below that header. Thus for tables, we add additional rules for converting rows into sentences: ● We apply a rule to match the table headers against the pattern ‘lot [token]’ where [token] is an optional word (e.g., ‘number’, ‘no.’); ● For the column headers matched by this rule, we find the one where all the texts of cells in that column are ‘number-like’ tokens. Then, we insert the header text before the number-like token within that cell. As an example, the row span with a value ‘1’ in Figure 2 will be updated to ‘Lot Number 1’ and will be repeated for both row 2 and 3 when creating a sentence. Some may argue that lot items detection from tables should be simpler as once a table is classified as a ‘relevant’ one containing lot and item descriptions, given our ‘horizontal table’ assumption, we can simply take each row (after ‘expanding’ row/column spans) as an instance of item description. While this may be true for many cases like that shown in Figure 2, practically, there are complex table structures such as that shown in Figure 7. Here, each lot is split into a consecutive number of rows, and within each lot, the headers are repeated for the item(s) in that lot. The actual content we wish to extract from the tables are those indicated in the box. For this reason, we opt for the generic approach described above, i.e., treating each row in a table as a sentence for classification. The same feature set and algorithm described above are used, and the training datasets will be explained later. Authors:
(1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk);
(2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io);
(3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io);
(4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk);
(5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). This paper is available on arxiv under CC BY 4.0 license. Table of Links Abstract and Introduction Domain and Task
2.1. Data sources and complexity
2.2. Task definition Related Work
3.1. Text mining and NLP research overview
3.2. Text mining and NLP in industry use
3.3. Text mining and NLP for procurement
3.4. Conclusion from literature review Proposed Methodology
4.1. Domain knowledge
4.2. Content extraction
4.3. Lot zoning
4.4. Lot item detection
4.5. Lot parsing
4.6. XML parsing, data joining, and risk indices development Experiment and Demonstration
5.1. Component evaluation
5.2. System demonstration Discussion
6.1. The ‘industry’ focus of the project
6.2. Data heterogeneity, multilingual and multi-task nature
6.3. The dilemma of algorithmic choices
6.4. The cost of training data Conclusion, Acknowledgements, and References Abstract and Introduction Abstract and Introduction Abstract and Introduction Abstract and Introduction Domain and Task 2.1. Data sources and complexity 2.2. Task definition Domain and Task Domain and Task 2.1. Data sources and complexity 2.1. Data sources and complexity 2.2. Task definition 2.2. Task definition Related Work 3.1. Text mining and NLP research overview 3.2. Text mining and NLP in industry use 3.3. Text mining and NLP for procurement 3.4. Conclusion from literature review Related Work Related Work 3.1. Text mining and NLP research overview 3.1. Text mining and NLP research overview 3.2. Text mining and NLP in industry use 3.2. Text mining and NLP in industry use 3.3. Text mining and NLP for procurement 3.3. Text mining and NLP for procurement 3.4. Conclusion from literature review 3.4. Conclusion from literature review Proposed Methodology 4.1. Domain knowledge 4.2. Content extraction 4.3. Lot zoning 4.4. Lot item detection 4.5. Lot parsing 4.6. XML parsing, data joining, and risk indices development Proposed Methodology Proposed Methodology Proposed Methodology 4.1. Domain knowledge 4.1. Domain knowledge 4.2. Content extraction 4.2. Content extraction 4.3. Lot zoning 4.3. Lot zoning 4.4. Lot item detection 4.4. Lot item detection 4.5. Lot parsing 4.5. Lot parsing 4.6. XML parsing, data joining, and risk indices development 4.6. XML parsing, data joining, and risk indices development Experiment and Demonstration 5.1. Component evaluation 5.2. System demonstration Experiment and Demonstration Experiment and Demonstration 5.1. Component evaluation 5.1. Component evaluation 5.2. System demonstration 5.2. System demonstration Discussion 6.1. The ‘industry’ focus of the project 6.2. Data heterogeneity, multilingual and multi-task nature 6.3. The dilemma of algorithmic choices 6.4. The cost of training data Discussion Discussion 6.1. The ‘industry’ focus of the project 6.1. The ‘industry’ focus of the project 6.2. Data heterogeneity, multilingual and multi-task nature 6.2. Data heterogeneity, multilingual and multi-task nature 6.3. The dilemma of algorithmic choices 6.3. The dilemma of algorithmic choices 6.4. The cost of training data 6.4. The cost of training data Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References Conclusion, Acknowledgements, and References 4.4. Lot item detection With the relevant pages and tables identified, our next step in the process is to identify the text elements in the pages/tables that actually describe lots and items. We would like to capture two kinds of information: the texts that indicate a lot reference and the texts that describe individual items in a lot. In practice, this is often contained in a single, coherent text passage, such as that shown in Figure 5. Thus we handle this as a binary sentence classification task where a sentence is a positive example if it contains either a lot reference, or item information, or both; and negative otherwise. We deter the extraction of lot reference and structured item information to the next component ‘lot parsing’. We also need to deal with unstructured texts from a page and content from tables in slightly different ways. 4.4.1. Unstructured texts in a page 4.4.1. Unstructured texts in a page For unstructured texts in a page, we apply sentence splitting and then sentence classification. For features, we adapt the 24 page-level features explained above in the ‘page selection’section by applying them to each sentence to classify. As an example, in equation 1, we replace 𝑝 with the input sentence 𝑠 and by alternating the 𝑚𝑒𝑡𝑟𝑖𝑐 we obtain 4 features. The same can be applied to equations 2~3. Then replacing 𝐿 with the other two domain dictionaries will produce another 8 features. Further, we add three binary features that help capture lot references: ● If the sentence contains the word ‘lot’ (and its translation in other languages) ● If the sentence contains the word ‘number’ or ‘no.’ or ‘num’ (and its translations) ● If the sentence contains number-like tokens (including roman/arabic numerals and patterns such as ‘1.1, 1.21, X.2, II.1’) We also compare the same set of algorithms for classification. 4.4.2. Tables 4.4.2. Tables Similar to the idea explained in passage selection, we simply treat each row as a sentence and therefore, the solution would be almost the same as that for dealing with unstructured texts above. One exception is for tables like that in Figure 2, where the lot references need to be interpreted based on combining the table header ‘Lot Number’ and the text in the row spans below that header. Thus for tables, we add additional rules for converting rows into sentences: ● We apply a rule to match the table headers against the pattern ‘lot [token]’ where [token] is an optional word (e.g., ‘number’, ‘no.’); ● For the column headers matched by this rule, we find the one where all the texts of cells in that column are ‘number-like’ tokens. Then, we insert the header text before the number-like token within that cell. As an example, the row span with a value ‘1’ in Figure 2 will be updated to ‘Lot Number 1’ and will be repeated for both row 2 and 3 when creating a sentence. Some may argue that lot items detection from tables should be simpler as once a table is classified as a ‘relevant’ one containing lot and item descriptions, given our ‘horizontal table’ assumption, we can simply take each row (after ‘expanding’ row/column spans) as an instance of item description. While this may be true for many cases like that shown in Figure 2, practically, there are complex table structures such as that shown in Figure 7. Here, each lot is split into a consecutive number of rows, and within each lot, the headers are repeated for the item(s) in that lot. The actual content we wish to extract from the tables are those indicated in the box. For this reason, we opt for the generic approach described above, i.e., treating each row in a table as a sentence for classification. The same feature set and algorithm described above are used, and the training datasets will be explained later. Authors: (1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk); (2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io); (3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io); (4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk); (5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). Authors: Authors: (1) Ziqi Zhang*, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Ziqi.Zhang@sheffield.ac.uk); (2) Tomas Jasaitis, Vamstar Ltd., London (Tomas.Jasaitis@vamstar.io); (3) Richard Freeman, Vamstar Ltd., London (Richard.Freeman@vamstar.io); (4) Rowida Alfrjani, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Rowida.Alfrjani@sheffield.ac.uk); (5) Adam Funk, Information School, the University of Sheffield, Regent Court, Sheffield, UKS1 4DP (Adam.Funk@sheffield.ac.uk). This paper is available on arxiv under CC BY 4.0 license. This paper is available on arxiv under CC BY 4.0 license. available on arxiv available on arxiv

Using Machine Learning for Lot and Item Identification in Tenders

SQL stands for structured query language. I’m not going to call SQL a programming language, as in my opinion, it’s not. SQL is a query language. That means that when you piece the query language together, it’s to retrieve data based on the rules that your query defines. There’s a lot (I repeat, a lot!) of uses for basic SQL. Software Engineers uses SQL, data science uses SQL, data analytics uses SQL, heck, even accountants use SQL. SQL skills and thought process can also be translated to a lot of different ETL based tasks. If you know the SQL basics, you can boost your employability for sure. You also don’t need a lot to start to learn. A decent laptop and a free mySQL install and you’re good to go. In my mind, learning of any programming related topic is not a sprint, it’s a marathon. With that in mind though, lets' look at some realistic timelines for learning SQL. Benefits of Learning SQL Structured query language (SQL) is common anywhere that requires database systems. And guess where database systems are used? Pretty much everywhere. If you are interested in data analytics, data science, software engineering, or database management, you will need to know SQL. Learning SQL can also lead to high-paying jobs that involve other programming languages. Factors Affecting Learning Speed for Programming Languages For starters, you need to be able to dedicate some time to learning. When I started as a Junior Software Engineer 20 years ago, I was fortunate enough to get paid to learn. I was able to spend hours reading, taking courses and practicing in real world experiences. If you can learn on the job, it’s certainly easier. You may be cross training from other programming languages, or from a completely different department. Either way, being in the proximity of a SQL developer aids the learning process, that’s for sure. If you are not learning on the job, there are lots of amazing course providers out there that offer courses in both basic SQL and advanced SQL skills. Without someone to guide you through though, factor in that this will take a little longer. Learning Structured Query Language (SQL): A Step-by-Step Guide Beginner Level: SQL Fundamentals Let’s quickly go through the basics: SELECT statement -- Select all columns from the 'users' table
SELECT * 
FROM users;

-- Select specific columns (e.g., 'id', 'name') from the 'users' table
SELECT id, name, email 
FROM users 
WHERE status = 'active'; INSERT statement -- Insert a single row into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES (1, 'John Doe', 'johndoe@example.com', 'active');

-- Insert multiple rows into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES 
(2, 'Jane Smith', 'janesmith@example.com', 'inactive'),
(3, 'Sam Wilson', 'samwilson@example.com', 'active'); UPDATE statement -- Update the 'status' column for a specific user
UPDATE users
SET status = 'inactive'
WHERE id = 1;

-- Update multiple columns for users with a certain condition
UPDATE users
SET email = 'newemail@example.com', status = 'active'
WHERE name = 'Jane Smith'; DELETE statement -- Delete a specific row based on a condition
DELETE FROM users
WHERE id = 1;

-- Delete all rows where status is 'inactive'
DELETE FROM users
WHERE status = 'inactive'; Learning these basic SQL concepts can be done in a few days. Intermediate Level: Advanced SQL Concepts How long does it take to learn SQL to an intermediate level? A year or two to do it properly. Intermediate sql skills here. SQL Functions, JOINs, Subqueries and CTEs. This is to learn how to manipulate data using SQL queries. JOINS INNER JOIN with a WHERE clause. -- Retrieve users and their corresponding orders
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
INNER JOIN orders ON users.id = orders.user_id
WHERE orders.amount > 100; This query joins the ‘users’ table and the ‘orders’ table where the ‘id’ in ‘users’ matches the ‘user_id’ in ‘orders’. LEFT JOIN -- Retrieve all users and their orders (include users with no orders)
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
LEFT JOIN orders ON users.id = orders.user_id; CTEs -- Use a CTE to retrieve users with high order amounts
WITH HighSpendingUsers AS (
    SELECT users.id, users.name, SUM(orders.amount) AS total_spent
    FROM users
    INNER JOIN orders ON users.id = orders.user_id
    GROUP BY users.id, users.name
    HAVING SUM(orders.amount) > 500
)
SELECT * 
FROM HighSpendingUsers;

-- Explanation:
-- 1. The CTE 'HighSpendingUsers' calculates the total spent by each user.
-- 2. Only users with a total order amount greater than 500 are included.
-- 3. The main query selects all results from the CTE. Subqueries -- Retrieve users and the total amount of their orders
SELECT 
    id, 
    name, 
    (SELECT SUM(amount) FROM orders WHERE orders.user_id = users.id) AS total_spent
FROM users; Practice with complex queries and data analysis using the above features and you’ll be well in to your intermediate SQL skills level. Advanced Level: Specialized Topics How long does it take to learn SQL to an advanced level? Probably years. Everyones learning process is different, but to truly master the above skill set and understand not only how to use the features, but when to use them should be achievable in 3 to 6 months depending on how many hours a day you are able to practice and learn. Let’s move on the advanced SQL functions. Stored Procedures A stored procedure is a collection of individual SQL statements placed together to make a reusable set of combined functionality. You would usually pass a few parameters in to the stored procedure, it would use those parameters to define the data required and then return you something hopefully useful. -- Create a stored procedure to fetch user details by user ID
CREATE PROCEDURE GetUserById
    @UserId INT
AS
BEGIN
    -- Select statement to return the user object
    SELECT id, name, email, status
    FROM users
    WHERE id = @UserId;
END; Above, we have a stored procedure. It takes userId as an input, and returns a row of data from the ‘users’ table, containing the columns: Id, name, email and status. To execute the stored procedure, we simply: -- Execute the stored procedure with a specific UserId
EXEC GetUserById @UserId = 1; and it returns (assuming the data is present): id    | name       | email              | status
1     | John Doe   | johndoe@mail.com   | active Database Performance Advanced users will need to take time to learn about database performance. Sometimes you will have a database administrator to help you with this, but other times as an engineer, you’ll be on your own. Advanced SQL then isn’t just about syntax, it’s about understanding how to write complex queries to manipulate data on a relational database. Effective Learning Strategies As always with programming languages, you need to start at the beginning and take your time. I learned on the job and picked a lot up as I went as and when I needed. My SQL skills came as I needed to solve new problems and manipulate data in ways I hadn’t before. Learning SQL for me was a mix of reading books (this was 20 years ago!), taking courses, pair programming and diving deep when coming across a stumbling block in a project. Practice as often as you can with real world scenarios. There’s nothing better than solving an actual problem rather than a contrived example. Get involved with communities. The microsoft sql server groups on LinkedIn are great. Friendly people with a lot of knowledge. So how Long Does It Take to Learn SQL? As always, questions like this are very hard to answer universally. The time it takes to learn SQL can really vary depending on your background, the time you get to spend learning and your aptitude towards this type of task. If you have no programming knowledge, learning very basic SQL skills can take up to around two weeks. Intermediate SQL skills can be picked up in the first few months of dedicated work, and I’d say that advanced SQL skills will take up to a year of solid work. If you are already familiar with other programming languages and have prior programming experience, or have used relational databases before, learning SQL could be much faster than that. Just remember, don’t be hard on yourself. It takes time and most people genuinely have what it takes to learn SQL. Just Another Tech Lead - Software Engienering, Tech and People Leadership X (Twitter)
YouTube SQL stands for structured query language. I’m not going to call SQL a programming language, as in my opinion, it’s not. SQL is a query language. That means that when you piece the query language together, it’s to retrieve data based on the rules that your query defines. There’s a lot (I repeat, a lot!) of uses for basic SQL. Software Engineers uses SQL, data science uses SQL, data analytics uses SQL, heck, even accountants use SQL. SQL skills and thought process can also be translated to a lot of different ETL based tasks. If you know the SQL basics, you can boost your employability for sure. You also don’t need a lot to start to learn. A decent laptop and a free mySQL install and you’re good to go. In my mind, learning of any programming related topic is not a sprint , it’s a marathon. With that in mind though, lets' look at some realistic timelines for learning SQL. sprint Benefits of Learning SQL Structured query language (SQL) is common anywhere that requires database systems. And guess where database systems are used? Pretty much everywhere. If you are interested in data analytics, data science, software engineering, or database management, you will need to know SQL. Learning SQL can also lead to high-paying jobs that involve other programming languages. Factors Affecting Learning Speed for Programming Languages For starters, you need to be able to dedicate some time to learning. When I started as a Junior Software Engineer 20 years ago, I was fortunate enough to get paid to learn. I was able to spend hours reading, taking courses and practicing in real world experiences. If you can learn on the job, it’s certainly easier. You may be cross training from other programming languages, or from a completely different department. Either way, being in the proximity of a SQL developer aids the learning process, that’s for sure. If you are not learning on the job, there are lots of amazing course providers out there that offer courses in both basic SQL and advanced SQL skills. Without someone to guide you through though, factor in that this will take a little longer. Learning Structured Query Language (SQL): A Step-by-Step Guide Beginner Level: SQL Fundamentals Let’s quickly go through the basics: SELECT statement -- Select all columns from the 'users' table
SELECT * 
FROM users;

-- Select specific columns (e.g., 'id', 'name') from the 'users' table
SELECT id, name, email 
FROM users 
WHERE status = 'active'; -- Select all columns from the 'users' table
SELECT * 
FROM users;

-- Select specific columns (e.g., 'id', 'name') from the 'users' table
SELECT id, name, email 
FROM users 
WHERE status = 'active'; INSERT statement -- Insert a single row into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES (1, 'John Doe', 'johndoe@example.com', 'active');

-- Insert multiple rows into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES 
(2, 'Jane Smith', 'janesmith@example.com', 'inactive'),
(3, 'Sam Wilson', 'samwilson@example.com', 'active'); -- Insert a single row into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES (1, 'John Doe', 'johndoe@example.com', 'active');

-- Insert multiple rows into the 'users' table
INSERT INTO users (id, name, email, status) 
VALUES 
(2, 'Jane Smith', 'janesmith@example.com', 'inactive'),
(3, 'Sam Wilson', 'samwilson@example.com', 'active'); UPDATE statement -- Update the 'status' column for a specific user
UPDATE users
SET status = 'inactive'
WHERE id = 1;

-- Update multiple columns for users with a certain condition
UPDATE users
SET email = 'newemail@example.com', status = 'active'
WHERE name = 'Jane Smith'; -- Update the 'status' column for a specific user
UPDATE users
SET status = 'inactive'
WHERE id = 1;

-- Update multiple columns for users with a certain condition
UPDATE users
SET email = 'newemail@example.com', status = 'active'
WHERE name = 'Jane Smith'; DELETE statement -- Delete a specific row based on a condition
DELETE FROM users
WHERE id = 1;

-- Delete all rows where status is 'inactive'
DELETE FROM users
WHERE status = 'inactive'; -- Delete a specific row based on a condition
DELETE FROM users
WHERE id = 1;

-- Delete all rows where status is 'inactive'
DELETE FROM users
WHERE status = 'inactive'; Learning these basic SQL concepts can be done in a few days. Intermediate Level: Advanced SQL Concepts How long does it take to learn SQL to an intermediate level? A year or two to do it properly. Intermediate sql skills here. SQL Functions, JOINs, Subqueries and CTEs. This is to learn how to manipulate data using SQL queries. JOINS INNER JOIN with a WHERE clause. -- Retrieve users and their corresponding orders
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
INNER JOIN orders ON users.id = orders.user_id
WHERE orders.amount > 100; -- Retrieve users and their corresponding orders
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
INNER JOIN orders ON users.id = orders.user_id
WHERE orders.amount > 100; This query joins the ‘users’ table and the ‘orders’ table where the ‘id’ in ‘users’ matches the ‘user_id’ in ‘orders’. LEFT JOIN -- Retrieve all users and their orders (include users with no orders)
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
LEFT JOIN orders ON users.id = orders.user_id; -- Retrieve all users and their orders (include users with no orders)
SELECT users.id, users.name, orders.order_id, orders.amount
FROM users
LEFT JOIN orders ON users.id = orders.user_id; CTEs -- Use a CTE to retrieve users with high order amounts
WITH HighSpendingUsers AS (
    SELECT users.id, users.name, SUM(orders.amount) AS total_spent
    FROM users
    INNER JOIN orders ON users.id = orders.user_id
    GROUP BY users.id, users.name
    HAVING SUM(orders.amount) > 500
)
SELECT * 
FROM HighSpendingUsers;

-- Explanation:
-- 1. The CTE 'HighSpendingUsers' calculates the total spent by each user.
-- 2. Only users with a total order amount greater than 500 are included.
-- 3. The main query selects all results from the CTE. -- Use a CTE to retrieve users with high order amounts
WITH HighSpendingUsers AS (
    SELECT users.id, users.name, SUM(orders.amount) AS total_spent
    FROM users
    INNER JOIN orders ON users.id = orders.user_id
    GROUP BY users.id, users.name
    HAVING SUM(orders.amount) > 500
)
SELECT * 
FROM HighSpendingUsers;

-- Explanation:
-- 1. The CTE 'HighSpendingUsers' calculates the total spent by each user.
-- 2. Only users with a total order amount greater than 500 are included.
-- 3. The main query selects all results from the CTE. Subqueries -- Retrieve users and the total amount of their orders
SELECT 
    id, 
    name, 
    (SELECT SUM(amount) FROM orders WHERE orders.user_id = users.id) AS total_spent
FROM users; -- Retrieve users and the total amount of their orders
SELECT 
    id, 
    name, 
    (SELECT SUM(amount) FROM orders WHERE orders.user_id = users.id) AS total_spent
FROM users; Practice with complex queries and data analysis using the above features and you’ll be well in to your intermediate SQL skills level. Advanced Level: Specialized Topics How long does it take to learn SQL to an advanced level? Probably years. Everyones learning process is different, but to truly master the above skill set and understand not only how to use the features, but when to use them should be achievable in 3 to 6 months depending on how many hours a day you are able to practice and learn. Let’s move on the advanced SQL functions. Stored Procedures A stored procedure is a collection of individual SQL statements placed together to make a reusable set of combined functionality. You would usually pass a few parameters in to the stored procedure, it would use those parameters to define the data required and then return you something hopefully useful. -- Create a stored procedure to fetch user details by user ID
CREATE PROCEDURE GetUserById
    @UserId INT
AS
BEGIN
    -- Select statement to return the user object
    SELECT id, name, email, status
    FROM users
    WHERE id = @UserId;
END; -- Create a stored procedure to fetch user details by user ID
CREATE PROCEDURE GetUserById
    @UserId INT
AS
BEGIN
    -- Select statement to return the user object
    SELECT id, name, email, status
    FROM users
    WHERE id = @UserId;
END; Above, we have a stored procedure. It takes userId as an input, and returns a row of data from the ‘users’ table, containing the columns: Id, name, email and status. To execute the stored procedure, we simply: -- Execute the stored procedure with a specific UserId
EXEC GetUserById @UserId = 1; -- Execute the stored procedure with a specific UserId
EXEC GetUserById @UserId = 1; and it returns (assuming the data is present): id    | name       | email              | status
1     | John Doe   | johndoe@mail.com   | active id    | name       | email              | status
1     | John Doe   | johndoe@mail.com   | active Database Performance Advanced users will need to take time to learn about database performance. Sometimes you will have a database administrator to help you with this, but other times as an engineer , you’ll be on your own. engineer Advanced SQL then isn’t just about syntax, it’s about understanding how to write complex queries to manipulate data on a relational database. Effective Learning Strategies As always with programming languages, you need to start at the beginning and take your time. I learned on the job and picked a lot up as I went as and when I needed. My SQL skills came as I needed to solve new problems and manipulate data in ways I hadn’t before. Learning SQL for me was a mix of reading books (this was 20 years ago!), taking courses, pair programming and diving deep when coming across a stumbling block in a project. Practice as often as you can with real world scenarios. There’s nothing better than solving an actual problem rather than a contrived example. Get involved with communities. The microsoft sql server groups on LinkedIn are great. Friendly people with a lot of knowledge. So how Long Does It Take to Learn SQL? As always, questions like this are very hard to answer universally. The time it takes to learn SQL can really vary depending on your background, the time you get to spend learning and your aptitude towards this type of task. If you have no programming knowledge, learning very basic SQL skills can take up to around two weeks. Intermediate SQL skills can be picked up in the first few months of dedicated work, and I’d say that advanced SQL skills will take up to a year of solid work. If you are already familiar with other programming languages and have prior programming experience, or have used relational databases before, learning SQL could be much faster than that. relational databases Just remember, don’t be hard on yourself. It takes time and most people genuinely have what it takes to learn SQL. Just Another Tech Lead - Software Engienering, Tech and People Leadership X (Twitter) YouTube X (Twitter) X (Twitter) YouTube YouTube

You Don't Need More Than 2 Weeks to Learn Basic SQL

Developer Relations in Web3: How Learning DevRel Helps Build a Career

Let's Build a Practical Home Lab for Learning and Experimentation

Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction
2. Related Work
3. SwitchLight and 3.1. Preliminaries
3.2. Problem Formulation
3.3. Architecture
3.4. Objectives
4. Multi-Masked Autoencoder Pre-training
5. Data
6. Experiments
7. Conclusion Appendix A. Implementation Details
B. User Study Interface
C. Video Demonstration
D. Additional Qualitative Results & References 3.3. Architecture Illum Net. The network infers the lighting conditions in the given image captured in an HDRI format. Specifically, it computes the convolved HDRIs: The network employs a cross-attention mechanism at its core, where predefined Phong reflectance lobes serve as queries, and the original image acts as both keys and values. Within this setup, the convolved HDRI maps are synthesized by integrating image information into the Phong reflectance lobe representation. Specifically, our model utilizes bottleneck features from the Normal Net as a compact image representation. Our approach simplifies the complex task of HDRI reconstruction by instead focusing on estimating interactions with known surface reflective properties. We have empirically validated that it significantly enhances albedo prediction across a range of real-world scenarios. Specular Net. The network infers surface attributes associated with the Cook-Torrance specular elements, specifically, the roughness α and Fresnel reflectivity f0. It uses a source image, predicted normal, and albedo maps as inputs. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Authors: Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 5 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.3. Architecture 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 5. Data 6. Experiments 7. Conclusion Abstract and 1. Introduction Abstract and 1. Introduction 2. Related Work 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.2. Problem Formulation 3.3. Architecture 3.3. Architecture 3.4. Objectives 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 4. Multi-Masked Autoencoder Pre-training 5. Data 5. Data 6. Experiments 6. Experiments 7. Conclusion 7. Conclusion Appendix A. Implementation Details B. User Study Interface C. Video Demonstration D. Additional Qualitative Results & References A. Implementation Details A. Implementation Details B. User Study Interface B. User Study Interface C. Video Demonstration C. Video Demonstration D. Additional Qualitative Results & References D. Additional Qualitative Results & References 3.3. Architecture Illum Net. The network infers the lighting conditions in the given image captured in an HDRI format. Specifically, it computes the convolved HDRIs : Illum Net. HDRIs The network employs a cross-attention mechanism at its core, where predefined Phong reflectance lobes serve as queries, and the original image acts as both keys and values. Within this setup, the convolved HDRI maps are synthesized by integrating image information into the Phong reflectance lobe representation. Specifically, our model utilizes bottleneck features from the Normal Net as a compact image representation. Our approach simplifies the complex task of HDRI reconstruction by instead focusing on estimating interactions with known surface reflective properties. We have empirically validated that it significantly enhances albedo prediction across a range of real-world scenarios. Specular Net. The network infers surface attributes associated with the Cook-Torrance specular elements, specifically, the roughness α and Fresnel reflectivity f0. It uses a source image, predicted normal, and albedo maps as inputs. Specular Net. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

AI Is Learning How to Perfect Lighting in Photos

Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 7 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction
2. Related Work
3. SwitchLight and 3.1. Preliminaries
3.2. Problem Formulation
3.3. Architecture
3.4. Objectives
4. Multi-Masked Autoencoder Pre-training
5. Data
6. Experiments
7. Conclusion Appendix A. Implementation Details
B. User Study Interface
C. Video Demonstration
D. Additional Qualitative Results & References 4. Multi-Masked Autoencoder Pre-training We introduce the Multi-Masked Autoencoder (MMAE), a self-supervised pre-training framework designed to enhance feature representations in relighting models. It aims to improve output quality without relying on additional, costly light stage data. Building upon the MAE framework [19], MMAE capitalizes on the inherent learning of crucial image features like structure, color, and texture, which are essential for relighting tasks. However, adapting MAE to our specific needs poses several non-trivial challenges. Firstly, MAE is primarily designed for vision transformers [15], while our focus is on a UNet, a convolution-based architecture. This convolutional structure, with its hierarchical nature and aggressive pooling, is known to simplify the MAE task, necessitating careful adaptation [50]. Further, the hyperparameters of MAE, particularly the fixed mask size and ratio, are also specific to vision transformers. These factors could introduce biases during training and hinder the model to recognize image features at various scales. Moreover, MAE relies solely on masked region reconstruction loss, limiting the model to understand the global coherence of the reconstructed region in relation to its visible context. To address these challenges effectively, we have developed two key strategies within the MMAE framework: Dynamic Masking. MMAE eliminates two key hyperparameters, mask size and ratio, by introducing a variety of mask types to generalize the MAE. These types, which include overlapping patches of various sizes, outpainting masks [46], and free-form masks [29] (see Fig.5), each contribute to the model’s versatility. With the ability to handle challenging masked regions, MMAE achieves a more comprehensive understanding of image properties. Generative Target. In addition to its structural advancements, MMAE incorporates a new loss function strategy. We have adopted perceptual [24] and adversarial losses [22], along with the original reconstruction loss. As a result, MMAE is equipped not only to reconstruct missing image parts but also to ensure synthesis capabilities and their seamless integration with the original context. In practice, the weights for the three losses are equally set. We pre-train the entire UNet architecture using MMAE, and, unlike MAE, we retain the decoder and fine-tune the entire model on relighting ground truths. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Authors: Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 7 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 7 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 7 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 7 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.3. Architecture 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 5. Data 6. Experiments 7. Conclusion Abstract and 1. Introduction Abstract and 1. Introduction 2. Related Work 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.2. Problem Formulation 3.3. Architecture 3.3. Architecture 3.4. Objectives 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 4. Multi-Masked Autoencoder Pre-training 5. Data 5. Data 6. Experiments 6. Experiments 7. Conclusion 7. Conclusion Appendix A. Implementation Details B. User Study Interface C. Video Demonstration D. Additional Qualitative Results & References A. Implementation Details A. Implementation Details B. User Study Interface B. User Study Interface C. Video Demonstration C. Video Demonstration D. Additional Qualitative Results & References D. Additional Qualitative Results & References 4. Multi-Masked Autoencoder Pre-training We introduce the Multi-Masked Autoencoder (MMAE), a self-supervised pre-training framework designed to enhance feature representations in relighting models. It aims to improve output quality without relying on additional, costly light stage data. Building upon the MAE framework [19], MMAE capitalizes on the inherent learning of crucial image features like structure, color, and texture, which are essential for relighting tasks. However, adapting MAE to our specific needs poses several non-trivial challenges. Firstly, MAE is primarily designed for vision transformers [15], while our focus is on a UNet, a convolution-based architecture. This convolutional structure, with its hierarchical nature and aggressive pooling, is known to simplify the MAE task, necessitating careful adaptation [50]. Further, the hyperparameters of MAE, particularly the fixed mask size and ratio, are also specific to vision transformers. These factors could introduce biases during training and hinder the model to recognize image features at various scales. Moreover, MAE relies solely on masked region reconstruction loss, limiting the model to understand the global coherence of the reconstructed region in relation to its visible context. To address these challenges effectively, we have developed two key strategies within the MMAE framework: Dynamic Masking. MMAE eliminates two key hyperparameters, mask size and ratio, by introducing a variety of mask types to generalize the MAE. These types, which include overlapping patches of various sizes, outpainting masks [46], and free-form masks [29] (see Fig.5), each contribute to the model’s versatility. With the ability to handle challenging masked regions, MMAE achieves a more comprehensive understanding of image properties. Dynamic Masking. Generative Target. In addition to its structural advancements, MMAE incorporates a new loss function strategy. We have adopted perceptual [24] and adversarial losses [22], along with the original reconstruction loss. As a result, MMAE is equipped not only to reconstruct missing image parts but also to ensure synthesis capabilities and their seamless integration with the original context. In practice, the weights for the three losses are equally set. Generative Target. We pre-train the entire UNet architecture using MMAE, and, unlike MAE, we retain the decoder and fine-tune the entire model on relighting ground truths. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

AI Learns to Perfect Lighting in Photos Using Smart Masks and Creative Training

Authors:
(1) Shehan Munasinghe, Mohamed bin Zayed University of AI and Equal Contribution;
(2) Rusiru Thushara, Mohamed bin Zayed University of AI and Equal Contribution;
(3) Muhammad Maaz, Mohamed bin Zayed University of AI;
(4) Hanoona Abdul Rasheed, Mohamed bin Zayed University of AI;
(5) Salman Khan, Mohamed bin Zayed University of AI and Australian National University;
(6) Mubarak Shah, University of Central Florida;
(7) Fahad Khan, Mohamed bin Zayed University of AI and Linköping University. Editor's Note: This is Part 3 of 10 of a study detailing the development of a smarter AI model for videos. Read the rest below. Table of Links Abstract and 1 Introduction
2. Related Works


PG-Video-LLaVA


3.1. Overview
3.2. Architecture


Experiments


4.1. Implementation Details
4.2. Stronger Baseline
4.3. Spatial Grounding in Videos
4.4. Zero-Shot Visual Question Answering
5. Conclusion and References Supplementary Material A. Audio Modality Integration
B. Visual Grounding: Quantitative Evaluation
C. Qualitative Results for Visual Grounding
D. Quantitative Evaluations of Video-based Conversation Performance 3.1. Overview In this paper, we introduce PG-Video-LLaVA, a novel Large Multimodal Model (LMM) designed to align video and audio representations with a Large Language Model (LLM). This integration equips PG-Video-LLaVA with the capability to proficiently manage both video and audio data in conversational contexts. Additionally, our method integrates a specialized plug-and-play module for effective video grounding (see Figure 2). In constructing PG-Video-LLaVA, our approach integrates sophisticated mechanisms for aligning video and audio signals with language processing capabilities, thereby facilitating a comprehensive multimodal analysis. Central to our model is an advanced CLIP-based video encoder, which has been specifically adapted to process both spatial and temporal dimensions of video data. This adaptation enables a deeper understanding of video content, setting PGVideo-LLaVA apart from conventional image-centric models. For training, PG-Video-LLaVA utilizes the VideoInstruct100K [22] dataset comprising 100K video instructions derived from ActivityNet-200 [11]. This diverse dataset ensures that the model is well-equipped to handle a broad spectrum of video contexts with high accuracy. In addition to visual processing, PG-Video-LLaVA incorporates stateof-the-art audio analysis by leveraging advanced audio transcription techniques, similar to those employed in WhisperX [2] and Whisper-AT[10]. This integration allows the model to process and understand audio inputs effectively, enhancing its overall multimodal interpretation capabilities. While PG-Video-LLaVA’s foundation is based on the LLaVA-1.5 [18] framework, it is extended for videos to incorporate spatio-temporal representations, audio understanding and visual grounding capabilities. Its unique combination of enhanced video encoding, extensive training dataset, integrated audio processing and grounding capability marks it as a forward step in the field of LMMs. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Shehan Munasinghe, Mohamed bin Zayed University of AI and Equal Contribution; (2) Rusiru Thushara, Mohamed bin Zayed University of AI and Equal Contribution; (3) Muhammad Maaz, Mohamed bin Zayed University of AI; (4) Hanoona Abdul Rasheed, Mohamed bin Zayed University of AI; (5) Salman Khan, Mohamed bin Zayed University of AI and Australian National University; (6) Mubarak Shah, University of Central Florida; (7) Fahad Khan, Mohamed bin Zayed University of AI and Linköping University. Authors: Authors: (1) Shehan Munasinghe, Mohamed bin Zayed University of AI and Equal Contribution; (2) Rusiru Thushara, Mohamed bin Zayed University of AI and Equal Contribution; (3) Muhammad Maaz, Mohamed bin Zayed University of AI; (4) Hanoona Abdul Rasheed, Mohamed bin Zayed University of AI; (5) Salman Khan, Mohamed bin Zayed University of AI and Australian National University; (6) Mubarak Shah, University of Central Florida; (7) Fahad Khan, Mohamed bin Zayed University of AI and Linköping University. Editor's Note: This is Part 3 of 10 of a study detailing the development of a smarter AI model for videos. Read the rest below. Editor's Note: This is Part 3 of 10 of a study detailing the development of a smarter AI model for videos. Read the rest below. Editor's Note: This is Part 3 of 10 of a study detailing the development of a smarter AI model for videos. Read the rest below. Editor's Note: This is Part 3 of 10 of a study detailing the development of a smarter AI model for videos. Read the rest below. Table of Links Abstract and 1 Introduction 2. Related Works PG-Video-LLaVA 3.1. Overview 3.2. Architecture Experiments 4.1. Implementation Details 4.2. Stronger Baseline 4.3. Spatial Grounding in Videos 4.4. Zero-Shot Visual Question Answering 5. Conclusion and References Abstract and 1 Introduction Abstract and 1 Introduction 2. Related Works 2. Related Works PG-Video-LLaVA PG-Video-LLaVA PG-Video-LLaVA 3.1. Overview 3.1. Overview 3.2. Architecture 3.2. Architecture Experiments Experiments Experiments 4.1. Implementation Details 4.1. Implementation Details 4.2. Stronger Baseline 4.2. Stronger Baseline 4.3. Spatial Grounding in Videos 4.3. Spatial Grounding in Videos 4.4. Zero-Shot Visual Question Answering 4.4. Zero-Shot Visual Question Answering 5. Conclusion and References 5. Conclusion and References Supplementary Material Supplementary Material A. Audio Modality Integration B. Visual Grounding: Quantitative Evaluation C. Qualitative Results for Visual Grounding D. Quantitative Evaluations of Video-based Conversation Performance A. Audio Modality Integration A. Audio Modality Integration B. Visual Grounding: Quantitative Evaluation B. Visual Grounding: Quantitative Evaluation C. Qualitative Results for Visual Grounding C. Qualitative Results for Visual Grounding D. Quantitative Evaluations of Video-based Conversation Performance D. Quantitative Evaluations of Video-based Conversation Performance 3.1. Overview In this paper, we introduce PG-Video-LLaVA, a novel Large Multimodal Model (LMM) designed to align video and audio representations with a Large Language Model (LLM). This integration equips PG-Video-LLaVA with the capability to proficiently manage both video and audio data in conversational contexts. Additionally, our method integrates a specialized plug-and-play module for effective video grounding (see Figure 2). In constructing PG-Video-LLaVA, our approach integrates sophisticated mechanisms for aligning video and audio signals with language processing capabilities, thereby facilitating a comprehensive multimodal analysis. Central to our model is an advanced CLIP-based video encoder, which has been specifically adapted to process both spatial and temporal dimensions of video data. This adaptation enables a deeper understanding of video content, setting PGVideo-LLaVA apart from conventional image-centric models. For training, PG-Video-LLaVA utilizes the VideoInstruct100K [22] dataset comprising 100K video instructions derived from ActivityNet-200 [11]. This diverse dataset ensures that the model is well-equipped to handle a broad spectrum of video contexts with high accuracy. In addition to visual processing, PG-Video-LLaVA incorporates stateof-the-art audio analysis by leveraging advanced audio transcription techniques, similar to those employed in WhisperX [2] and Whisper-AT[10]. This integration allows the model to process and understand audio inputs effectively, enhancing its overall multimodal interpretation capabilities. While PG-Video-LLaVA’s foundation is based on the LLaVA-1.5 [18] framework, it is extended for videos to incorporate spatio-temporal representations, audio understanding and visual grounding capabilities. Its unique combination of enhanced video encoding, extensive training dataset, integrated audio processing and grounding capability marks it as a forward step in the field of LMMs. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Stories, News & Mentions about LEAR

Lear’s Stories on HackerNoon

Lear's News & Mentions