Get the most recent info and news about carbonTRACK on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

carbonTRACK

carbontrack

The Master of Information Management and Systems (MIMS) program educates information professionals to provide leadership for an information-driven world. The Master of Information and Data Science (MIDS) is an online degree preparing data science ...

CarbonTrack: Visualizing AI Upscaled Ecosystem Dynamics

This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Authors: (1) Minghao Yan, University of Wisconsin-Madison; (2) Hongyi Wang, Carnegie Mellon University; (3) Shivaram Venkataraman, myan@cs.wisc.edu. Table of Links Abstract & Introduction Motivation Opportunities Architecture Overview Proble Formulation: Two-Phase Tuning Modeling Workload Interference Experiments Conclusion & References A. Hardware Details B. Experimental Results C. Arithmetic Intensity D. Predictor Analysis 8 CONCLUSION In this work, we examine the unique characteristics of energy consumption in neural network inference, especially for edge devices. We identified unique tradeoffs and dimensions between energy consumption and inference latency SLOs and empirically demonstrated hidden components in optimizing energy consumption. We then propose an optimization framework that automatically and holistically tunes various hardware components to find a configuration aligned with the Pareto Frontier. We empirically verify the effectiveness and efficiency of PolyThrottle. PolyThrottle also adapts to the need for fine-tuning and proposes a simple performance prediction model to adaptively schedule finetuning requests while keeping the online inference workload under the inference latency SLO whenever possible. We hope our study sheds more light on the hidden dimension of NN energy optimization. REFERENCES Jetson partner solutions ebook. URL https://resources.nvidia.com/ en-us-jetson-success-stories/ jetson-partner-solutions-ebook?lx=XRDs_y. Twitter streaming traces, 2018. URL https://archive.org/details/ archiveteam-twitter-stream-201804. Alipourfard, O., Liu, H. H., Chen, J., Venkataraman, S., Yu, M., and Zhang, M. Cherrypick: Adaptively unearthing the best cloud configurations for big data analytics. In NSDI, volume 2, pp. 4–2, 2017. Anderson, T., Belay, A., Chowdhury, M., Cidon, A., and Zhang, I. Treehouse: A case for carbon-aware datacenter software. 2022. Anthony, L. F. W., Kanding, B., and Selvan, R. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. 2020. Arafa, Y., ElWazir, A., ElKanishy, A., Aly, Y., Elsayed, A., Badawy, A., Chennupati, G., Eidenbenz, S., and Santhi, N. Verified instruction-level energy consumption measurement for nvidia gpus. 2020. Bai, Z., Zhang, Z., Zhu, Y., and Jin, X. Pipeswitch: Fast pipelined context switching for deep learning applications. In Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation, pp. 499–514, 2020. Banner, R., Hubara, I., Hoffer, E., and Soudry, D. Scalable methods for 8-bit training of neural networks. Advances in neural information processing systems, 31, 2018. Brochu, E., Cora, V. M., and De Freitas, N. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599, 2010. Cai, H., Wang, T., Wu, Z., Wang, K., Lin, J., and Han, S. On-device image classification with proxyless neural architecture search and quantization-aware fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, pp. 0–0, 2019. Cai, H., Gan, C., Zhu, L., and Han, S. Tinytl: Reduce memory, not parameters for efficient on-device learning. Advances in Neural Information Processing Systems, 33: 11285–11297, 2020. Cao, Q., Balasubramanian, A., and Balasubramanian, N. Towards accurate and reliable energy measurement of nlp models. In Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing, 2020. Censor, Y. Pareto optimality in multiobjective problems. Applied Mathematics and Optimization, 4(1):41–59, 1977. Courbariaux, M., Bengio, Y., and David, J.-P. Training deep neural networks with low precision multiplications. arXiv preprint arXiv:1412.7024, 2014. Courbariaux, M., Bengio, Y., and David, J.-P. Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in neural information processing systems, 28, 2015. Crankshaw, D., Wang, X., Zhou, G., Franklin, M. J., Gonzalez, J. E., and Stoica, I. Clipper: A low latency online prediction serving system. In NSDI, volume 17, pp. 613– 627, 2017. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. Gardner, J. R., Kusner, M. J., Xu, Z. E., Weinberger, K. Q., and Cunningham, J. P. Bayesian optimization with inequality constraints. In ICML, volume 2014, pp. 937–945, 2014. Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., and Keutzer, K. A survey of quantization methods for efficient neural network inference. arXiv preprint arXiv:2103.13630, 2021. Gog, I., Kalra, S., Schafhalter, P., Gonzalez, J. E., and Stoica, I. D3: a dynamic deadline-driven approach for building autonomous vehicles. In Proceedings of the Seventeenth European Conference on Computer Systems, pp. 453–471, 2022. Gu, D., Xie, X., Huang, G., Jin, X., and Liu, X. Energyefficient gpu clusters scheduling for deep learning. arXiv preprint arXiv:2304.06381, 2023. Gujarati, A., Karimi, R., Alzayat, S., Hao, W., Kaufmann, A., Vigfusson, Y., and Mace, J. Serving {DNNs} like clockwork: Performance predictability from the bottom up. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pp. 443–462, 2020. Gupta, U., Kim, Y. G., Lee, S., Tse, J., Lee, H.-H. S., Wei, G.-Y., Brooks, D., and Wu, C.-J. Chasing carbon: The elusive environmental footprint of computing. IEEE Micro, 42(4):37–47, 2022. He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., Qiu, H., et al. Fedml: A research library and benchmark for federated machine learning. arXiv preprint arXiv:2007.13518, 2020. Hodak, M., Gorkovenko, M., and Dholakia, A. Towards power efficiency in deep learning on data center hardware. In IEEE International Conference on Big Data, 2019. Hong, S. and Kim, H. An integrated gpu power and performance model. In ISCA, 2010. Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., et al. Searching for mobilenetv3. pp. 1314–1324, 2019. Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017. Ivanov, A., Dryden, N., Ben-Nun, T., Li, S., and Hoefler, T. Data movement is all you need: A case study on optimizing transformers. Proceedings of Machine Learning and Systems, 3:711–732, 2021. Kandasamy, K., Vysyaraju, K. R., Neiswanger, W., Paria, B., Collins, C. R., Schneider, J., Poczos, B., and Xing, E. P. Tuning hyperparameters without grad students: Scalable and robust bayesian optimisation with dragonfly. The Journal of Machine Learning Research, 21(1):3098–3124, 2020. Kandiah, V., Peverelle, S., Khairy, M., Pan, J., Manjunath, A., Rogers, T. G., Aamodt, T. M., and Hardavellas, N. Accelwattch: A power modeling framework for modern gpus. In MICRO, 2021. Kang, D.-K., Lee, K.-B., and Kim, Y.-C. Cost efficient gpu cluster management for training and inference of deep learning. Energies, 15(2):474, 2022. Kim, S., Gholami, A., Yao, Z., Mahoney, M. W., and Keutzer, K. I-bert: Integer-only bert quantization. pp. 5506–5518. PMLR, 2021. Klein, A., Bartels, S., Falkner, S., Hennig, P., and Hutter, F. Towards efficient bayesian optimization for big data. In NIPS 2015 Bayesian Optimization Workshop, 2015. Komoda, T., Hayashi, S., Nakada, T., Miwa, S., and Nakamura, H. Power capping of cpu-gpu heterogeneous systems through coordinating dvfs and task mapping. In 2013 IEEE 31st International Conference on computer design (ICCD). IEEE, 2013. Lacoste, A., Luccioni, A., Schmidt, V., and Dandres, T. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700, 2019. Lane, N. D. and Georgiev, P. Can deep learning revolutionize mobile sensing? In Proceedings of the 16th international workshop on mobile computing systems and applications, pp. 117–122, 2015. Lee, J., Chirkov, N., Ignasheva, E., Pisarchyk, Y., Shieh, M., Riccardi, F., Sarokin, R., Kulik, A., and Grundmann, M. On-device neural net inference with mobile gpus. arXiv preprint arXiv:1907.01989, 2019. Lowe-Power, J., Ahmad, A. M., Akram, A., Alian, M., Amslinger, R., Andreozzi, M., Armejach, A., Asmussen, N., Beckmann, B., Bharadwaj, S., et al. The gem5 simulator: Version 20.0+. arXiv preprint arXiv:2007.03152, 2020. Mei, X., Wang, Q., and Chu, X. A survey and measurement study of gpu dvfs on energy conservation. Digital Communications and Networks, 3(2):89–100, 2017. Nabavinejad, S. M., Reda, S., and Ebrahimi, M. Batchsizer: Power-performance tradeoff for dnn inference. In Proceedings of the 26th Asia and South Pacific Design Automation Conference, 2021. NVIDIA. Stream management, 2023a. URL https://docs.nvidia.com/cuda/ cuda-runtime-api. NVIDIA. Nvidia multi-instance gpu, 2023b. URL https://docs.nvidia.com/datacenter/ tesla/mig-user-guide/index.html. Peng, Y., Zhu, Y., Chen, Y., Bao, Y., Yi, B., Lan, C., Wu, C., and Guo, C. A generic communication scheduler for distributed dnn training acceleration. In SOSP, 2019. Qiao, A., Choe, S. K., Subramanya, S. J., Neiswanger, W., Ho, Q., Zhang, H., Ganger, G. R., and Xing, E. P. Pollux: Coadaptive cluster scheduling for goodput-optimized deep learning. In OSDI, 2021. Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. Squad: 100,000+ questions for machine comprehension of text. In EMNLP, 2016. Romero, F., Li, Q., Yadwadkar, N. J., and Kozyrakis, C. {INFaaS}: Automated model-less inference serving. In 2021 USENIX Annual Technical Conference (USENIX ATC 21), pp. 397–411, 2021. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019. Schwartz, R., Dodge, J., Smith, N. A., and Etzioni, O. Green ai. Commun. ACM, 63(12):54–63, 2020. Shen, H., Chen, L., Jin, Y., Zhao, L., Kong, B., Philipose, M., Krishnamurthy, A., and Sundaram, R. Nexus: A gpu cluster engine for accelerating dnn-based video analysis. In Proceedings of the 27th ACM Symposium on Operating Systems Principles, pp. 322–337, 2019. Snoek, J., Larochelle, H., and Adams, R. P. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012. Strubell, E., Ganesh, A., and McCallum, A. Energy and policy considerations for deep learning in nlp. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. Tambe, T., Hooper, C., Pentecost, L., Jia, T., Yang, E.- Y., Donato, M., Sanh, V., Whatmough, P., Rush, A. M., Brooks, D., et al. Edgebert: Sentence-level energy optimizations for latency-aware multi-task nlp inference. In MICRO, 2021. Tan, M. and Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. pp.6105–6114. PMLR, 2019. Tang, Z., Wang, Y., Wang, Q., and Chu, X. The impact of gpu dvfs on the energy and performance of deep learning: An empirical study. In Proceedings of the Tenth ACM International Conference on Future Energy Systems, 2019. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jegou, H. Training data-efficient image transform- ´ ers & distillation through attention. pp. 10347–10357. PMLR, 2021. Venkataraman, S., Yang, Z., Franklin, M., Recht, B., and Stoica, I. Ernest: Efficient performance prediction for {Large-Scale} advanced analytics. In 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 16), pp. 363–378, 2016. Wan, C., Santriaji, M., Rogers, E., Hoffmann, H., Maire, M., and Lu, S. Alert: Accurate learning for energy and timeliness. In ATC, 2020. Wang, F., Zhang, W., Lai, S., Hao, M., and Wang, Z. Dynamic gpu energy optimization for machine learning training workloads. IEEE Transactions on Parallel and Distributed Systems, 2021. Wang, G., Venkataraman, S., Phanishayee, A., Devanur, N., Thelin, J., and Stoica, I. Blink: Fast and generic collectives for distributed ml. In Proceedings of Machine Learning and Systems, 2020a. Wang, Y., Wang, Q., Shi, S., He, X., Tang, Z., Zhao, K., and Chu, X. Benchmarking the performance and energy efficiency of ai accelerators for ai training. In 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), 2020b. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. Transformers: State-of-the-art natural language processing. In EMNLP, 2020. Wu, C.-J., Raghavendra, R., Gupta, U., Acun, B., Ardalani, N., Maeng, K., Chang, G., Aga, F., Huang, J., Bai, C., Gschwind, M., Gupta, A., Ott, M., Melnikov, A., Candido, S., Brooks, D., Chauhan, G., Lee, B.,Lee, H.-H., Akyildiz, B., Balandat, M., Spisak, J., Jain, R., Rabbat, M., and Hazelwood, K. Sustainable ai: Environmental implications, challenges and opportunities. In Proceedings of Machine Learning and Systems, 2022. Wu, X., Rao, J., Chen, W., Huang, H., Ding, C., and Huang, H. Switchflow: preemptive multitasking for deep learning. In Proceedings of the 22nd Inter Xu, M., Liu, J., Liu, Y., Lin, F. X., Liu, Y., and Liu, X. A first look at deep learning apps on smartphones. In The World Wide Web Conference, WWW ’19, pp. 2125–2136, 2019. You, J., Chung, J.-W., and Chowdhury, M. Zeus: Understanding and optimizing gpu energy consumption of dnn training. arXiv preprint arXiv:2208.06102, 2022. Yu, P. and Chowdhury, M. Fine-grained gpu sharing primitives for deep learning applications. Proceedings of Machine Learning and Systems, 2:98–111, 2020. Zhao, Y., Liu, X., Liu, S., Li, X., Zhu, Y., Huang, G., Liu, X., and Jin, X. Muxflow: Efficient and safe gpu sharing in large-scale production deep learning clusters. arXiv preprint arXiv:2303.13803, 2023.

PolyThrottle: Energy-efficient Neural Network Inference on Edge Devices: Conclusion & References

What Are the Top Startups in Oceania?

Flow is a permissionless layer-1 blockchain built to support the high-scale use cases of games, virtual worlds, and the digital assets that power them. The blockchain was created by the team behind Cryptokitties, Dapper Labs, and NBA Top Shot. One core attribute that differentiates Flow from the other blockchains is its way of providing to objects within the blockchain. Fine-grained access is a new method of providing access to users that follows the principle of least privilege. This makes it highly secure and fault-resistant. Flow is one of the unique blockchains to have this, as it is built on the fundamentals of Capability-based security. fine-grained permissions But how does this help you as a developer? If you’re writing smart contracts, then you likely have to regularly define access to objects and functions within your contract. Understanding the different types of access better will help you to define access types in code more accurately. This will allow you to write highly secure and efficient smart contracts. Let’s dive in. Understanding fine-grained permissions As opposed to Role-based access control (RBAC), where permissions are defined loosely for every role and not specifically for every individual, fine-grained permissions allow developers to define permissions for objects at the lowest level. This allows permissions to be defined for every object. This also allows you to add new permissions types, such as time-expiry and geography-based permissions. Let’s understand the core principle behind fine-grained permissions. The principle of least privilege The concept of least privilege means that in a computing environment, any module (such as a process, a user, or a programmer) only has the necessary access that it needs to do its specific job. Any other unnecessary permissions for its specific job are either revoked or unavailable. For example, a programmer whose job is to create backups does not need to make user data updates, so it should only have read permissions on the database of users. This principle is the core principle behind fine-grained permissions. Capability-based Security A capability is an unforgeable token of authority. In capability-based authorization, your identity does not matter. If you’ve been sent an access token by the owner/admin that grants you the capability to access a resource and you can execute that capability, then you will have access. Hence, at runtime, the application does not check what your identity is but only that you can access the requested resource. Thus, fine-grained access can be implemented on the foundations of capability-based security. History of fine-grained access In the 1980s, operating systems would define permissions on objects as “read”, “write”, and “execute”. This was called Access Control Lists. In the 1990s, Role-based Access Control was introduced, which allowed you to create groups and assign users to groups. Then, the concepts of attribute-based access control and capability-based access control arrived, which brought fine-grained access into the picture. In the world of blockchains, however, fine-grained access is new and brought by Cadence, the language used to write Flow smart contracts. Fine-grained access in Cadence Cadence, the language used to write flow smart contracts, has built-in special keywords to allow for fine-grained permissions to resources. Below, we dive into the common keywords and the usage of these keywords to provide fine-grained access. The access keywords Flow defines certain keywords, known as access keywords, with different parameters to define the access privileges of a certain resource. Below are the four primary keywords: Pub or access(all) Using this keyword means the declaration is accessible in all scopes. However, this also means that the declaration is not updatable except when you’re updating the value at an index of an array. Pub or access(all) are the least restrictive types of access and should generally be avoided. Below is an example of using the pub keyword: pub contract Car {    pub let carBrand: String    pub let carNameByBrand: {String: String}    pub fun returnCarBrand(): String {        return self.carBrand    }}} Following the above code example, if there’s an object called CarObject of type Car, then the statement below invocation is valid. CarObject.carBrand //Equals the value of greeting in the object The values with access type pub are generally not updatable except when updating the value at a certain index of an array. CarObject.carNameByBrand = {} //Not ValidCarObject.carNameByBrand[“Porsche”] = “Carrera” //Valid access(self) access(self) means that the declaration is only visible in the current and inner scopes. For example, an access(self) field can only be accessed by the functions of the type it is a part of. When using access(self), it is common practice to define setters and getter methods to allow other objects and contracts to access this field. This keyword makes the usage very safe but also restrictive. Below is an example of using access(self): pub contract Car {    access(self) let carBrand: String    access(self) let carNameByBrand: {String: String}    pub fun returnCarBrand(): String {        return self.carBrand    }    pub fun returnCarName(_ brand: String): String? {        return self.carNameByBrand[brand]    }}} access(contract) access(contract) means that the declaration is only accessible in the scope of the contract that defines it. Hence, functions defined in other contracts under the same account or functions defined in contracts under other accounts will not be able to access it. This keyword is unique to Cadence and comes in handy when you have any contract-specific variables you don’t want to expose to other contracts. Below is an example of using the access(contract) keyword: pub contract CarContract {    pub struct Car {        access(contract) var carBrand: String    }    pub fun returnCarBrand(_brand: Car): String {        return Car.carBrand    }}} In the above example, if you have a method in a different contract that has an object of the Car struct, you cannot get the carBrand variable directly. However, you can call the returnCarBrand function to get the value of carBrand. access(account) This keyword means that the declaration is only accessible inside the entire account owned by the current smart contract. Hence, other contracts in the same account can access a variable freely. This type of access is very powerful, allowing you to keep a variable private to your account. You are also able to deploy a smart contract that needs to use the variable in the future, if needed. This way, several contracts in the same account can share information seamlessly with one another, promoting the modularity of code. Below is an example of how to use it: pub contract CarContract {    pub struct Car {        access(account) var carBrand: String    }    pub fun returnCarBrand(_brand: Car): String {        return Car.carBrand    }}} In the above example, if you have a method in a different contract that has an object of the Car struct, you can get the carBrand variable directly. Conclusion The concept of least privilege and capability-based security play a significant role in Flow's precise permissions. The access keywords provide clear definitions of variable and function access, as we have covered in this article. One of the most noteworthy features that fundamentally strengthens the security of the Flow blockchain is its fine-grained access control. Blockchain technology has long been plagued by bugs and hacks, most of which occur because of ill-conceived access controls. However, fine-grained access control provides developers with a remarkable opportunity to effortlessly and smoothly secure their smart contracts. Ultimately, this is what sets Flow apart in the world of blockchain. On the topic of fine-grained permissions, hopefully you found this in-depth exploration valuable and helpful. For more information, refer to the . Or explore the and get some hands-on experience! official documentation Flow Docs

Better Smart Contracts with Fine-Grained Permissions using Cadence

In this guide, we will use Jenkins as a Continuous Integration server and Nexus as a build repository. The end goal of this guide is to create a flow, where we can build, store, organize and monitor the compiled artifacts by Maven through our CI server. Before we begin, make sure that you already have Jenkins up and running. In case you haven’t set up Jenkins yet, then copy the below command and run it on your Docker enabled host/machine. $ docker run -d --name jenkins-ci -p : jenkins/jenkins:lts 8080 8080 Once, Jenkins container is configured on your local/remote machine. Go to your preferred browser and open the URL . http:///your-ip-addr:8080 On the very first page, Jenkins will ask you for the admin password, which you can find by running below mentioned command in your terminal: $  docker exec -i jenkins-ci cat / /jenkins_home/secrets/initialAdminPassword

b5102c8d9fa245dbb0b8da03f504d3a5 var Follow the guided steps to finish the configuration. Save the username and password securely for future use. Installing Nexus: Nexus is a repository manager that allows you to store and retrieve artifacts. It enables you to host your built artifacts in a private and secure repo. You can always pull the Nexus Docker image using the following command: $ docker pull sonatype/nexus3
 
Using tag: latest
latest: Pulling sonatype/nexus3
cb3c77f9bdd8: Pull complete 
fd8daf2668d1: Pull complete 
fd1ff82b00e8: Pull complete a05f7b573af: Pull complete 
Digest: sha256: dfbc3eb094fe5cbbacec87aa8b91d16394dab627177e1deeebb5ac8ee
Status: Downloaded newer image sonatype/nexus3:latest
docker.io/sonatype/nexus3:latest default from 2 6570855 for Now it’s time to run downloaded sonatype/nexus on the default port 8081. Follow the below-mentioned commands: $ docker run -d --name nexus_repo -p : sonatype/nexus3 8081 8081 Usually it takes 1 to 2 minutes for the Nexus service to launch in your newly created Docker container. If you wish to follow the log to see if Nexus is up and ready, then run the following command: $ docker logs nexus_repo -f In logs you will see a message stating . It means your Nexus Repository Manager is ready to use. Now go to your browser and open . Started Sonatype Nexus OSS 3.20.1-01 http://your-ip-addr:8081 Find the option as shown below: Sign In The default username is admin, whereas to retrieve the password you need to run the following command: $ docker exec -i nexus_repo cat /nexus-data/admin.password ace93 f0d d2 b3b3a88d149 502 -5450 -4 -97 -9 And that’s it. Your Nexus Repository Manager is ready-to-use. The next step is to create a new repository. Create a Repository in Nexus: In this step, you are going to create a Maven Hosted repository in Nexus, where your Jenkins is going to upload “build” artifacts. Follow the below-mentioned steps to create a hosted repository, name it as , which you are going to use throughout this guide. Step 1: maven-nexus-repo Select recipe from the list as shown in the below-mentioned figure: maven2 (hosted) On the page, Step 2: Create Repository Enter the name as maven-nexus-repo In Version Policy, select the type of artifacts. Mixed Under the section, in , select . It will allow you to deploy an application multiple times. Hosted Deployment policy Allow redeploy To create a new user, go to . Select user type which happens to be the default Realm: Step 3: Dashboard > Server Administrator and Configuration > User > Create user Local In the page, Create User : Enter the desired ID; in our case, it is jenkins-user. ID : Enter the desired first name; in our case, it is Jenkins. First Name : Enter the desired second name; in our case, it is User. Last Name : Enter your email address. Email : Select from your drop-down menu. Status Active : Make sure that you grant the role to your user. Roles nx-admin In case you want more details for user creation, then . click here With this, we are through with the setup part of Nexus Repository Manager. Let us move to Jenkins to setup Nexus there. Install and Configure Nexus Plugins in Jenkins Here you are going to install and configure a few plugins for Nexus in Jenkins. For this, go to Jenkins and then and search and install and . Dashboard > Manage Jenkins > Manage Plugins > Available Nexus Artifact Uploader Pipeline Utility Steps Add Nexus Repository Manager’s user credentials in Jenkins. Go to , as shown below: Dashboard > Credentials > System > Global credentials (unrestricted) Next, set up Maven as a managed tool. Go to and find . Under this section, click on the button and add , as shown below: Dashboard > Manage Jenkins > Global Tool Configuration Maven Maven Installations Maven Alternatively, you can also install the Maven binary directly to your container on the directory. /var/jenkins_home Create a Jenkins Pipeline It’s time to create a Jenkins Job. Here you are going to use Pipeline job type, named as , as shown below: JenkinsNexus In the next page, find the Pipeline section and copy the below-mentioned script in the text area: pipeline {
    agent {
        label }
    tools {
        maven }
    environment {
        NEXUS_VERSION = NEXUS_PROTOCOL = NEXUS_URL = NEXUS_REPOSITORY = NEXUS_CREDENTIAL_ID = }
    stages {
        stage( ) {
            steps {
                script {
                    git ;
                }
            }
        }
        stage( ) {
            steps {
                script {
                    sh }
            }
        }
        stage( ) {
            steps {
                script {
                    pom = readMavenPom file: ;
                    filesByGlob = findFiles(glob: );
                    echo artifactPath = filesByGlob[ ].path;
                    artifactExists = fileExists artifactPath; (artifactExists) {
                        echo ;
                        nexusArtifactUploader(
                            nexusVersion: NEXUS_VERSION, : NEXUS_PROTOCOL, : NEXUS_URL, : pom.groupId, : pom.version, : NEXUS_REPOSITORY, : NEXUS_CREDENTIAL_ID, : [
                                [artifactId: pom.artifactId, : , : artifactPath, : pom.packaging],
                                [artifactId: pom.artifactId, : , : , : ]
                            ]
                        );
                    } {
                        error ;
                    }
                }
            }
        }
    }
} "master" "Maven" "nexus3" "http" "you-ip-addr-here:8081" "maven-nexus-repo" "nexus-user-credentials" "Clone code from VCS" 'https://github.com/javaee/cargotracker.git' "Maven Build" "mvn package -DskipTests=true" "Publish to Nexus Repository Manager" "pom.xml" "target/*.${pom.packaging}" "${filesByGlob[0].name} ${filesByGlob[0].path} ${filesByGlob[0].directory} ${filesByGlob[0].length} ${filesByGlob[0].lastModified}" 0 if "*** File: ${artifactPath}, group: ${pom.groupId}, packaging: ${pom.packaging}, version ${pom.version}" protocol nexusUrl groupId version repository credentialsId artifacts classifier '' file type classifier '' file "pom.xml" type "pom" else "*** File: ${artifactPath}, could not be found" Let’s break down the above-mentioned parameters bit by bit: : Here, we have to mention the exact version of Nexus, which can be or . In our case, it is latest version of . NEXUS_VERSION nexus2 nexus3 nexus3 : For this guide we have used HTTP protocol, although, in case of production, you will have to use HTTPS. NEXUS_PROTOCOL : Add your IP address and port number, where you are running Nexus. Make sure that you add Nexus instance details without mentioning protocols, i.e., or . NEXUS_URL https http : Enter the user ID, which you previously created in Jenkins, which in our case is . NEXUS_CREDENTIAL_ID nexus-user-credentials : Under stages, we used Project Git https://github.com/javaee/cargotracker As you are through with the Pipeline set up, it’s time to our project. Go to the project job page and click . As this is your first build, it is going to take some time, so sit tight. Build JenkinsNexus Build Now Once build is a success, in your Jenkins Console Output, you will see something like this: Whereas, in your Nexus Repository Manager, you would see something similar to this: Conclusion: A systematic way to disturb the project’s artifact is critical for any organization. With the help of Jenkins Pipeline and Nexus Repository Manager, you can centralize artifact repo, which ultimately reduces the efforts of reproducing build time as well as switch CI tools without worrying about migrating our artifacts. Nexus can be configured with cloud storage services like AWS S3 and Google Cloud Storage as well, which gives you additional freedom and delivery application without any hassle of storage. Hope you liked the article and took good use of it. You might also want to explore Configuring a CI/CD pipeline through Jenkins through . this article This article was originally published on https://appfleet.com/

Stories, News & Mentions about CARBONTRACK

Carbontrack’s Stories on HackerNoon

Carbontrack's News & Mentions