Дисертації з теми "Sécurité de la CI/CD"

Cette thèse explore les approches basées sur les données pour l'analyse automatique des causes profondes des échecs de construction dans les systèmes d'intégration continue et de déploiement continu (CI/CD), en se concentrant sur l'identification des échecs non déterministes, la localisation des messages de cause profonde dans les journaux de construction, et la caractérisation de la performance et de la sécurité des systèmes CI/CD. Basée sur des ensembles de données publics et industriels, nous explorons les propriétés des flux de travail CI/CD, telles que les temps d'exécution et les modèles d'échec. La recherche introduit l'utilisation du traitement du langage naturel (NLP) et des embeddings de graphes de connaissances (KGE) pour classifier les échecs de construction avec une précision de 94%. De plus, nous introduisons ChangeMyMind, une nouvelle méthode basée sur les réseaux neuronaux récurrents (RNNs) pour localiser avec précision les messages de cause profonde dans les journaux de construction sans étiquetage préalable des messages de cause profonde. Nous proposons X-Ray-TLS, une approche générique et transparente pour inspecter le trafic réseau chiffré TLS dans les environnements CI/CD. Enfin, la thèse revisite également les vulnérabilités de sécurité dans les systèmes CI/CD, démontrant le potentiel de compromissions à long terme indétectables. Ce travail a abouti à trois publications et deux soumissions en cours de révision, contribuant de manière significative à l'analyse et à l'optimisation des systèmes CI/CD
This thesis explores data-driven approaches for automated root cause analysis of CI/CD build failures, focusing on identifying non-deterministic failures, locating root cause messages in build logs, and characterizing CI/CD systems' performance and security. Grounded on public and industrial datasets, we explore CI/CD workflow properties, such as execution times and failure patterns. The research introduces the use of Natural Language Processing (NLP) and Knowledge Graphs Embeddings (KGE) for classifying build failures with a 94% accuracy. Additionally, we introduce ChangeMyMind, a new method based on Recurrent Neural Networks (RNNs) to accurately locate root cause messages in build logs without prior labeling of root cause messages. We propose X-Ray-TLS, a generic and transparent approach for inspecting TLS-encrypted network traffic in CI/CD environments. Finally, the thesis also revisits security vulnerabilities in CI/CD systems, demonstrating the potential for undetectable long-term compromises. This work has resulted in three publications and two under-review submissions, contributing significantly to CI/CD system analysis and optimization

The procedure of testing the implemented software is important and should be an essential and integrated part of the development process. In order for the testing to be meaningful it is important that the testing procedure ensures that the developed software meet certain requirements. The testing procure is often controlled by some sort of test specification. For many companies it is desirable to automate this procure. The focus of this thesis has been to automate a small subpart of the manual tests today performed related to SAAB:s air traffic management system. The automation has been achieved by studying the existing test specification which involves a lot of manual operations and to write software that mimics a few of these test cases. The thesis has resulted in a test framework which automates a small subset of the manual tests performed today. The framework has been designed to be scalable and to easily allow more test cases to be added by the personnel when time permits. The test framework has also been integrated with SAAB:s existing CI/CD workflow.

Multirepo model přístupu ke správě a verzování zdrojového kódu, jež zahrnuje použití mnoha oddělených repozitářů verzovacích systémů, je poslední dobou často zmiňován v odborné literatuře. Jednou z jeho nevýhod je množství zdlouhavých, nezajímavých a repetitivních úkonů, které je nutno provádět při hromadných operacích tvořících transakce napříč těmito repozitáři. Multirepo repozitáře navíc umožňují využití široké škály technologií, což jen umocňuje riziko lidské chyby, ke které při ručně prováděných hromadných operacích může dojít. V rámci této práce je navrženo, implementováno a otestováno řešení pro automatizaci operací prováděných napříč množstvím repozitářů uspořádaných v multirepo modelu, což s nimi uživatelům zlepšuje zkušenost.

Under VT 2019 ägde projektet rum varav denna rapport är ett av resultaten. Projektets mål var att skapa en CI/CD pipeline vars syfte var tänkt att frekvent kunna leverera färdigtestad kod till olika molntjänster som Google Cloud Platform, Amazon Web Services och Azure. Projektspecifikationerna gavs av företaget Skira för att skapa en snabbare integrationsprocess för nya utvecklare. Detta så en ny utvecklare skulle kunna lägga mer tid på att koda istället för att gräva ner sig i leverans-/testningsprocessen. Slutprodukten ger företag möjligheten att koda direkt på sitt utvecklingskluster.

Táto práca je prípadovou štúdiou postupného vývoja a nasadzovania lokačného softwaru v reálnom čase. Cieľom tejto práce je zrýchliť tento proces. Zvolený problém bol vyriešený s konvenčnými testovacími nastrojmi, vlastným nástrojom pre generovanie sieťovej prevádzky lokalizačnej platformy a nástrojmi CI/CD Gitlab. Prínosom tejto práce je zrýchlenie vývoja, zaručenie kvality vyvijaného softwaru a predstavenie spôsobu ako platformu pre lokalizáciu v reálnom čase testovať.

The focus of this study is on examining when employees in the IT industry experience that it is more favorable to use automated tests and manual tests, respectively. The purpose of this study is to investigate how different companies in practice, use, work with and think about the two different test methods. Four factors that influence the choice of test method have been developed as a workframe based on five articles, all of which discuss the requirements and criteria for the two different test methods. By conducting an interview study with semi-structured interviews, data has been retrieved from two different companies. The data has in turn been analyzed based on the four factors that have been developed, namely: the number of test cases/test runs, technical aspects, what functions that are to be tested and resources. Based on the analyzes, it has been clear that the opinions from the different respondents and from previous research often remain on the same track. Thus, the various criteria could be discussed and the motivation for when the respondents use each method could be outlined. However, it also becomes clear that in the end it is mainly resources, often the number of working hours and the monetary cost, that determines what practice of testing is to be used.
Denna studie riktar in sig på att undersöka när anställda inom IT-branschen upplever att det är mer gynnsamt att använda sig av automatiserade tester respektive manuella tester. Syftet med studien är att undersöka hur olika företag använder sig, arbetar med och ser på de två olika testmetoderna i praktiken. Fyra stycken faktorer som påverkar valet av testmetod har tagits fram som ramverk som är baserad på fem artiklar där samtliga diskuterar krav och kriterier för de två olika testmetoderna. Genom att utföra en intervjustudie med semi-strukturerade intervjuer har data samlats in från två olika företag. Den insamlade datan har i sin tur analyserats utifrån de olika faktorerna som tagits fram nämligen: Antalet testfall/testkörningar, tekniska aspekter, funktioner som ska testas samt resurser. Utifrån analysen har det varit tydligt att åsikterna från de olika respondenterna samt från tidigare forskning ofta är på samma spår. Därmed har de olika kriterierna kunnat diskuterats och motiveringarna till när respondenterna använder sig av respektive metod kunnat benas ut. Dock blir det också tydligt att i slutändan är det resurser, oftast antalet arbetstimmar och den monetära kostnaden, som styr vad som borde väljas.

L’azienda multinazionale Travelport Digital Ltd è una software house americana focalizzata sulle travel agencies che ha come obiettivo principale quello di semplificare la complessa industria dei trasporti e del turismo, fornendo servizi dedicati ai viaggiatori e ad organizzazioni terze. Il tirocinio di sei mesi presso la filiale di Dublino è stato speso lavorando in un Agile team e rivestendo il ruolo di Quality Assurance Engineer della componente Web frontend di un applicativo che aiuta le aziende a trovare le migliori opzioni di viaggio facendo scraping di voli, hotel, treni ed auto a noleggio. TripSource è un sistema di pianificazione e prenotazione unificata rivolto a business travellers che vogliono avere un itinerario dettagliato dei propri viaggi con update in tempo reale sullo stato dei mezzi prenotati. In particolare, mi sono occupato di svariate attività di sviluppo della test suite per l’applicativo TripSource: ho sviluppato test case per un progetto di User Interface Test Automation, migliorando la coverage della test suite end-to-end sulla componente grafica dell’applicativo Web; ho svolto attività di Manual Exploratory Testing sulle features appena terminate; ho contribuito alla scrittura di script per l’efficientamento della pipeline Continuous Integration e Continuous Delivery aziendale e per l’integrazione dei test automatici sviluppati. Per tutti i team di Travelport il testing e l’attenzione per la qualità sono componenti fondamentali del processo di sviluppo, ingredienti senza i quali il delivery non può avvenire. Il presente lavoro di tesi consiste quindi nello studio dell’attività di testing effettuata durante il periodo di tirocinio. In particolare si vuole approfondire design ed implementazione del progetto di Test Automation realizzato, fornendo anche una panoramica su come avviene l’integrazione del testing all’interno del complesso ciclo di sviluppo caratterizzante una realtà di business di dimensioni medio grandi.

Выпускная квалификационная работа 56 страниц, 19 рисунков, 11 источников, 8 приложений. Цель работы – разработка клиентской части веб-приложения «Мониторинг IT-конференций». В процессе работы был проведён анализ популярных фреймворков для веб-разработки, настроена интеграция с серверами CDN на базе сервиса Surge, создан Docker-образ с веб-приложением, настроена интеграция с GitHub Actions для CI/CD, настроен клиентский и серверный мониторинги на базе Sentry. В результате ВКР разработана клиентская часть на базе фреймворка Next.js для веб-приложения «Мониторинг IT-конференций».
Final qualification work 56 pages, 19 figures, 11 sources, 8 appendices. The purpose of the work is to develop the client part of the web application "Monitoring of IT conferences". In the process, we analyzed popular frameworks for web development, configured integration with CDN servers based on the Surge service, created a Docker image with a web application, configured integration with GitHub Actions for CI/CD, configured client and server monitoring based on Sentry. As a result of the final qualifying work, the client part was developed on the basis of the Next framework.js for the IT Conference Monitoring web application.

The amount of data being collected is increasing astronomically. Hence questions about privacy and data security are becoming more important than ever. A fast-changing culture is also reflected in the demands and requirements placed on software systems. Products and services need to evolve with the demands and feedback from customers to stay relevant on the market. Working methods and technologies have been refined to afford updating software continuously. However, rapidly changing software cause concern for the quality and level of security in the release. This thesis is a comprehensive literature study, reviewing the challenges of ensuring secure practises for continuously evolving software. The problem solved by the thesis is lack of an overall picture of the security concerns during continuous evolution. The findings are summarised in a checklist of areas of concern for security when maintaining and updating systems with continuous practises in cloud environments. This study shows that ensuring security, while delivering continuous releases, is a daunting task. It requires close collaboration between teams handling different aspects of software. This, in turn, entails a widening of competences to include knowledge about the work of other departments. It is concluded that personnel with this wide range of skill will be hard to acquire.
I en tid då mängden data som samlas in om individer ökar i ohindrad takt, blir frågor om integritet och informationssäkerhet viktigare än någonsin. Kraven på snabb utveckling och förändring präglar även metoderna för mjukvaruutveckling. Produkter och tjänster måste konstant anpassas efter kundernas önskemål för att förbli relevant på marknaden. Arbetssätt och teknologier har utvecklats över tid för att möjliggöra mjukvara som uppdateras kontinuerligt. Konstant föränderlig mjukvara leder dock till oro för kvalitén och säkerheten av uppdateringarna. Den här uppsatsen är en litteraturstudie som undersöker utmaningarna att säkerställa säkerhet för mjukvara som uppdateras kontinuerligt. Problemet som löses genom studien är den saknade helhetsbilden av säkerhetsproblem vid kontinuerligt föränderlig mjukvara. Resultatet sammanfattas i en checklista för områden som väcker oro för säkerheten vid arbetssätt som tillåter kontinuerliga uppdateringar i moln-miljöer. Studien visar att leverera säkra lösningar kontinuerligt är en svår uppgift. Det kräver nära samarbete mellan team som sköter olika delar av mjukvaruutveckling. Detta fordrar vida kompetenser som inkluderar förståelse av varandras arbete. Att finna personal med tillräckligt vida kompetenser uppskattas vara problematiskt.

This paper examines how adding security tools to a software pipeline affect the build time. Software development is an ever-changing field in a world where computers are trusted with almost everything society does. Meanwhile keeping build time low is crucial, and some aspects of quality assurance have therefore been left on the cutting room floor, security being one of the most vital and time-consuming. The time taken to scan for vulnerabilities has been suggested as a reason for the absence of security tests. By implementing nine different security tools into a generic DevOps pipeline, this paper aimed to examine the build times quantitatively.              The tools were selected using the OWASP Top Ten, coupled with an ISO standard, as a guideline. OWASP Juice Shop was used as the testing environment, and the scans managed to find most of the vulnerabilities in the Vulnerable Web Application. The pipeline was set up in Microsoft Azure and was configured in .yaml files. The resulting scan durations show that adding security measures to a build pipeline can add as little as 1/3 of the original build time.

Agile software development methods keep increasing in popularity. Many organizations who are using traditional software development methods, such as water-fall and stagegate based methods are switching to agile software development methods. This transition can be challenging, especially for organizations using project governance models that hinder the adoption of agile practices. This study aims to provide guidance to managers on how to introduce agile software development methods in such traditional organizations. The study is a single-case study on a large governmental agency in Sweden. Eight interviews with developers, team-leads and managers were conducted. The study identifies practical tools and ideas that managers can use to introduce a shared definition of agile, adopting an agile mindset, dedicated teams, and CI&CD. Together, this guidance supports managers with the introduction of agile software development methods in organizations utilizing traditional project governance structures and traditional software development methods.
Agila systemutvecklingsmetoder fortsätter öka i popularitet. Många organisationer som använder sig av traditionella systemutvecklingsmetoder så som vattenfallsmodellen byter till agila systemutvecklingsmetoder. Denna övergång kan vara utmanande, speciellt för organisationer som använder sig av projektbaserade förvaltningsmodeller som hindrar implementeringen av agila metoder. Den här studien syftar till att ge vägledningen till chefer kring hur de kan introducera agila systemutvecklingsmetoder iden nyss nämnda typen av traditionella organisationer. Studien är en fallstudie gjort på en stor myndighet i Sverige. Åtta intervjuer med systemutvecklare, lag-ledare och chefer har utförts. Studien identifierar verktyg och idéer som chefer kan använda sig avför att introducera en delad gemensam definition av agilt, anamma ett agilt tankesätt, introducera dedikerade teams och CI&CD. Tillsammans hjälper de här verktygen med introduktionen av agila systemutvecklingsmetoder i organisationer som använder sig av traditionella systemutvecklingsmetoder och förvaltningsmodeller.

Both distributed work and DevOps are on an upward trend. There is a slight resemblance between the problems that DevOps is trying to find answers to, the solutions, and the common problems that geographically distributed work faces. Mainly, they are related to isolated environments that have difficulties in mutual understanding and communication, collaboration. All this leads to inefficiencies and costs that affect the overall efficiency of companies. This report identifies how DevOps engineering principles and implementations provide solutions to common problems in globally distributed work environments. It uses a literature systematic literature search and review to extract the recent and relevant academic data in the scope of the two research questions. Then, a proof-of-concept is implemented for DevOps, which confirms the literature. In parallel, a survey addressed to Swedish companies provides subject-related data from the professional environment, which largely supports the literature and brings extra knowledge. All of this is considered in data analysis and formulation of conclusions, showing DevOps features that can improve and support work in globally distributed environments and outlining the importance of the tailored organizational culture for the modern need of large-scale distributed work.

DevOps is a cultural idea rather than a firm way to do software development, with the aim of reducing software lead times by bringing operations and development closer via principles that mainly deal with automation. This paper provides a potential DevOps solution for Wexnet, an internet service provider company. A requirements list is created by interviewing which is then used to evaluate existing web-based git solutions. Two viable candidates were selected, GitHub and GitLab which were compared against each other. GitLab was chosen because of its comparably low resource usage and lower overall setup complexity.

Práca sa zaoberá automatizáciou testovania technickej dokumentácie napísanej v značkovacom jazyku AsciiDoc pomocou open-source frameworku testovania technickej dokumentácie Emender implementovaného na CI/CD platforme. Framework bol rozšírený o webovú aplikáciu emenderwebservice s REST API, ktorá poskytuje užívateľské grafické rozhranie s výsledkami testov a mechanizmom na odrieknutie falošne pozitívnych výsledkov testov. Webová aplikácia bola vytvorená pomocou WSGI frameworku na tvorbu webových aplikácií Flask s databázou ktorá umožňuje agregáciu výsledkov testov a ich unikátnu identifikáciu. Aplikácia uľahčuje prístup ku výsledkom testov vygenerovaných frameworkom Emender v CI/CD systémoch a poskytuje technical writer-om ucelené užívateľské prostredie.

The rising of the DevOps movement and the transition from a product economy to a service economy drove significant changes in the software development life cycle paradigm, among which the dropping of the waterfall in favor of agile methods. Since DevOps is itself an agile method, it allows us to monitor current releases, receiving constant feedback from clients, and improving the next software releases. Despite its extraordinary development, DevOps still presents limitations concerning security, which needs to be included in the Continuous Integration or Continuous Deployment pipelines (CI/CD) used in software development. The massive adoption of cloud services and open-source software, the widely spread containers and related orchestration, as well as microservice architectures, broke all conventional models of software development. Due to these new technologies, packaging and shipping new software is done in short periods nowadays and becomes almost instantly available to users worldwide. The usual approach to attach security at the end of the software development life cycle (SDLC) is now becoming obsolete, thus pushing the adoption of DevSecOps or SecDevOps, by injecting security into SDLC processes earlier and preventing security defects or issues from entering into production. This dissertation aims to reduce the impact of microservices’ vulnerabilities by examining the respective images and containers through a flexible and adaptable set of analysis tools running in dedicated CI/CD pipelines. This approach intends to provide a clean and secure collection of microservices for later release in cloud production environments. To achieve this purpose, we have developed a solution that allows programming and orchestrating a battery of tests. There is a form where we can select several security analysis tools, and the solution performs this set of tests in a controlled way according to the defined dependencies. To demonstrate the solution’s effectiveness, we program a battery of tests for different scenarios, defining the security analysis pipeline to incorporate various tools. Finally, we will show security tools working locally, which subsequently integrated into our solution return the same results.
A ascensão da estratégia DevOps e a transição de uma economia de produto para uma economia de serviços conduziu a mudanças significativas no paradigma do ciclo de vida do desenvolvimento de software, entre as quais o abandono do modelo em cascata em favor de métodos ágeis. Uma vez que o DevOps é parte integrante de um método ágil, permite-nos monitorizar as versões actuais, recebendo feedback constante dos clientes, e melhorando as próximas versões de software. Apesar do seu extraordinário desenvolvimento, o DevOps ainda apresenta limitações relativas à segurança, que necessita de ser incluída nas pipelines de integração contínua ou implantação contínua (CI/CD) utilizadas no desenvolvimento de software. A adopção em massa de serviços na nuvem e software aberto, a ampla difusão de contentores e respectiva orquestração bem como das arquitecturas de micro-serviços, quebraram assim todos os modelos convencionais de desenvolvimento de software. Devido a estas novas tecnologias, a preparação e expedição de novo software é hoje em dia feita em curtos períodos temporais e ficando disponível quase instantaneamente a utilizadores em todo o mundo. Face a estes fatores, a abordagem habitual que adiciona segurança ao final do ciclo de vida do desenvolvimento de software está a tornar-se obsoleta, sendo crucial adotar metodologias DevSecOps ou SecDevOps, injetando a segurança mais cedo nos processos de desenvolvimento de software e impedindo que defeitos ou problemas de segurança fluam para os ambientes de produção. O objectivo desta dissertação é reduzir o impacto de vulnerabilidades em micro-serviços através do exame das respectivas imagens e contentores por um conjunto flexível e adaptável de ferramentas de análise que funcionam em pipelines CI/CD dedicadas. Esta abordagem pretende fornecer uma coleção limpa e segura de micro-serviços para posteriormente serem lançados em ambientes de produção na nuvem. Para atingir este objectivo, desenvolvemos uma solução que permite programar e orquestrar uma bateria de testes. Existe um formulário onde podemos seleccionar várias ferramentas de análise de segurança, e a solução executa este conjunto de testes de uma forma controlada de acordo com as dependências definidas. Para demonstrar a eficácia da solução, programamos um conjunto de testes para diferentes cenários, definindo as pipelines de análise de segurança para incorporar várias ferramentas. Finalmente, mostraremos ferramentas de segurança a funcionar localmente, que posteriormente integradas na nossa solução devolvem os mesmos resultados.
Mestrado em Engenharia Informática

This report is made within the Curricular Unit (UC) Project, in the 2nd year of the Master in Cyber-security and Forensic Informatics (MCIF) provided by the Polytechnic Institute of Leiria (IPL). The purpose of this project is to study SQL Injection vulnerabilities in web applications. According to OWASP (Open Web Application Security Project) [20][19], this is one of the more prevalent attacks on web applications. As part of this work a web application was implemented, which can from a URL address, go through all the endpoints of the target application and test for SQL Injection vulnerabilities. The application also makes allows for scheduling of the tests and it is integrable with Continuous Integration / Continuous Delivery (CI/CD) environments. According to the literature on the subject, there are several algorithms that can be employed to test for existing SQL Injection vulnerabilities in a web application. In this document, we analyze them both from a theoretical and an implementation point of view. In order to better understand the subject, and produce a useful tool in this space. With the development of this project, we concluded that it is possible to integrate SQL vulnerability tests, with CI/CD pipeline and automate the development process of an application, with the execution of SQL injection tests in an automated way.

Метою та завданням атестаційної роботи є удосконалення сайту кафедри програмної інженерії, створеного на системі WordPress. В процесі виконання роботи було проведено дослідження методології CI/CD, впровадження системи контролю версій та сучасних методик розробки веб-сайтів в цілому. В результаті розробки було оновлено систему створення сторінок сайту, адаптовано існуючі сторінки до перегляду на мобільних пристроях та встановлено сайт на систему контролю версій.
The purpose and task of the certification work is to improve the website of the Department of Software Engineering, created on the WordPress system. In the course of the work, a study of the CI / CD methodology, the introduction of a version control system and modern methods of website development in general was conducted. As a result of the development, the website creation system was updated, the existing pages were adapted for mobile browsing, and a version control system website was installed

This report will describe in detail all the work that resulted from the internship on xgeeks portugal, LDA. This internship was in the context of the 2nd year of master's degree in Computer Engineering - Mobile Computing, from Polytechnic of Leiria, and it had as theme "Architecture of large-scale React Native apps for Industry". The work done, was applied to two production apps, from two really different industries, the first for the food industry, and the second one for the fashion retail industry. But besides the difference, this document will hugely focus on the architecture, some high-level components of these systems and quality related characteristics, as testing and Continuous Integration and Delivery. The problem that was solved in these two companies is what they share the most, they were trying to use React Native for building their mobile apps, but without the proper architecture it wasn't able to scale. Therefore, in short, was built a custom solution on top of Redux that can handle a large-scale system with a large scale of users without breaking, all of it with proper quality assurance procedures and best practices.

Neste documento descreve-se o estágio curricular desenvolvido no âmbito da Unidade Curricular de Estágio do Mestrado de Engenharia Informática - Computação Móvel, pertencente ao Instituto Politécnico de Leiria, e realizado no Grupo Lusiaves SGPS,SA, no período compreendido entre Outubro de 2019 e Junho de 2020. Durante o período de estágio o objetivo foi de desenvolver mecanismos de Continuous Integration e Continuous Development, assim como o desenho e migração de aplicações monolíticas para containers . Adicionalmente, foi também essencial desenvolver aptidões relativas à Plataforma de Integrações utilizada na Empresa (Boomi), assim como aptidões ao nível de infraestrutura de empresa (AWS). Durante a duração do estágio e com recurso a uma metodologia ágil, foi possível implementar ferramentas, tecnologias e métodos que permitiram concretizar o objetivo inicial de se ter processos de Continuous Integration, Continuous Development e de contentores escaláveis. Foi possível implementar algumas soluções que ajudaram não só na qualidade dos dados de logging, como a solução de contentorização pretendida, assim como mecanismos que simplificaram o Continuous Integration e o Continuous Development.