{"id":190414,"date":"2023-02-20T17:59:00","date_gmt":"2023-02-20T17:59:00","guid":{"rendered":"https:\/\/datascientest.com\/?p=190414"},"modified":"2023-11-09T13:54:02","modified_gmt":"2023-11-09T13:54:02","slug":"apache-spark-streaming-tout-savoir","status":"publish","type":"post","link":"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir","title":{"rendered":"Spark Streaming : Qu&rsquo;est-ce que c&rsquo;est ? Comment \u00e7a fonctionne ?"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"190414\" class=\"elementor elementor-190414\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t\t<div class=\"elementor-element elementor-element-27fefdd e-flex e-con-boxed e-con\" data-id=\"27fefdd\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-995d81d elementor-widget elementor-widget-heading\" data-id=\"995d81d\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-heading-title{padding:0;margin:0;line-height:1}.elementor-widget-heading .elementor-heading-title[class*=elementor-size-]>a{color:inherit;font-size:inherit;line-height:inherit}.elementor-widget-heading .elementor-heading-title.elementor-size-small{font-size:15px}.elementor-widget-heading .elementor-heading-title.elementor-size-medium{font-size:19px}.elementor-widget-heading .elementor-heading-title.elementor-size-large{font-size:29px}.elementor-widget-heading .elementor-heading-title.elementor-size-xl{font-size:39px}.elementor-widget-heading .elementor-heading-title.elementor-size-xxl{font-size:59px}<\/style><h2 class=\"elementor-heading-title elementor-size-default\">Spark Streaming est une solution innovante pour le traitement en temps r\u00e9el des donn\u00e9es. Il s\u2019agit d\u2019un composant du framework Apache Spark qui offre une performance, une scalabilit\u00e9 et une fiabilit\u00e9 exceptionnelles.<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a54c9b4 elementor-widget elementor-widget-text-editor\" data-id=\"a54c9b4\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-text-editor.elementor-drop-cap-view-stacked .elementor-drop-cap{background-color:#69727d;color:#fff}.elementor-widget-text-editor.elementor-drop-cap-view-framed .elementor-drop-cap{color:#69727d;border:3px solid;background-color:transparent}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap{margin-top:8px}.elementor-widget-text-editor:not(.elementor-drop-cap-view-default) .elementor-drop-cap-letter{width:1em;height:1em}.elementor-widget-text-editor .elementor-drop-cap{float:left;text-align:center;line-height:1;font-size:50px}.elementor-widget-text-editor .elementor-drop-cap-letter{display:inline-block}<\/style>\t\t\t\t<p><span style=\"font-weight: 400;\">Ce <strong>syst\u00e8me de traitement en temps r\u00e9el distribu\u00e9<\/strong> a \u00e9t\u00e9 con\u00e7u pour r\u00e9pondre aux exigences les plus complexes en mati\u00e8re de traitement de donn\u00e9es en temps r\u00e9el. Il permet d\u2019effectuer des analyses complexes et des t\u00e2ches de transformations sur des donn\u00e9es provenant de diff\u00e9rentes sources (comme les r\u00e9seaux sociaux, les p\u00e9riph\u00e9riques connect\u00e9s ou les capteurs)<\/span><\/p><p><span style=\"font-weight: 400;\">Gr\u00e2ce \u00e0 ses fonctionnalit\u00e9s avanc\u00e9es, telles que la gestion des flux de donn\u00e9es tr\u00e8s important, l\u2019int\u00e9gration de data sources vari\u00e9es et la prise en charge de la tol\u00e9rance aux pannes, Spark Streaming s\u2019est impos\u00e9 comme <strong>un choix de r\u00e9f\u00e9rence pour les entreprises qui cherchent \u00e0 traiter efficacement des donn\u00e9es en temps r\u00e9el<\/strong>.<\/span><\/p><p><span style=\"font-weight: 400;\">Les domaines d\u2019application d\u2019une telle technologie sont tr\u00e8s vari\u00e9s. Nous pouvons citer par exemple la d\u00e9tection de fraude, la surveillance de march\u00e9s financiers, ou encore de la recommandation personnalis\u00e9e pour des achats en ligne, sans oublier bien entendu des analyses sur les r\u00e9seaux sociaux.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-0eccac6 e-flex e-con-boxed e-con\" data-id=\"0eccac6\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c570abc elementor-widget elementor-widget-heading\" data-id=\"c570abc\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Comment fonctionne le streaming de donn\u00e9es ?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a119acd elementor-widget elementor-widget-text-editor\" data-id=\"a119acd\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Le streaming de donn\u00e9es est un processus en temps r\u00e9el qui consiste \u00e0 traiter des donn\u00e9es qui sont g\u00e9n\u00e9r\u00e9es en continu et \u00e0 les analyser en temps r\u00e9el.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">Spark Streaming utilise une architecture dite \u00ab\u00a0micro-batch\u00a0\u00bb, ce qui signifie que les donn\u00e9es sont divis\u00e9es en petits lots appel\u00e9s \u00ab\u00a0batchs\u00a0\u00bb et sont trait\u00e9es de mani\u00e8re s\u00e9quentielle. Chaque batch est trait\u00e9 comme un <\/span><b>RDD<\/b><span style=\"font-weight: 400;\"> (Resilient Distributed Dataset) dans Spark, ce qui permet de <strong>b\u00e9n\u00e9ficier de la puissance de traitement en parall\u00e8le de Spark<\/strong>. Pour rappel, un RDD est l\u2019unit\u00e9 de base de donn\u00e9es dans <a href=\"https:\/\/datascientest.com\/apache-spark\">Apache Spark<\/a>, qui est une collection immuable de donn\u00e9es partag\u00e9es parall\u00e8lement sur plusieurs n\u0153uds d\u2019un cluster.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2709ef7 elementor-widget elementor-widget-image\" data-id=\"2709ef7\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<style>\/*! elementor - v3.15.0 - 02-08-2023 *\/\n.elementor-widget-image{text-align:center}.elementor-widget-image a{display:inline-block}.elementor-widget-image a img[src$=\".svg\"]{width:48px}.elementor-widget-image img{vertical-align:middle;display:inline-block}<\/style>\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1071\" height=\"239\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image2-1.png\" class=\"attachment-full size-full wp-image-190417\" alt=\"\" loading=\"lazy\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image2-1.png 1071w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image2-1-300x67.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image2-1-1024x229.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image2-1-768x171.png 768w\" sizes=\"(max-width: 1071px) 100vw, 1071px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-adbc047 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"adbc047\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/formations-data\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Apprendre \u00e0 faire du data streaming avec Spark<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-4007602 e-flex e-con-boxed e-con\" data-id=\"4007602\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-fcd454c elementor-widget elementor-widget-heading\" data-id=\"fcd454c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Qu'est ce que les DStreams ?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f1aad55 elementor-widget elementor-widget-text-editor\" data-id=\"f1aad55\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Spark Streaming fournit une abstraction de haut niveau appel\u00e9e \u00ab\u00a0<\/span><b>flux discr\u00e9tis\u00e9<\/b><span style=\"font-weight: 400;\">\u00a0\u00bb (<\/span><b>Discretized Stream<\/b><span style=\"font-weight: 400;\">), ou <\/span><b>DStream<\/b><span style=\"font-weight: 400;\">, qui repr\u00e9sentent un flux continu de donn\u00e9es. Ils peuvent \u00eatre consid\u00e9r\u00e9s comme des RDD continus, o\u00f9 chaque RDD repr\u00e9sente les donn\u00e9es g\u00e9n\u00e9r\u00e9es dans un intervalle d\u00e9fini.<\/span><\/p><p><span style=\"font-weight: 400;\">Les DStream peuvent \u00eatre cr\u00e9\u00e9s \u00e0 partir de flux d\u2019entr\u00e9e venant de sources telles que <a href=\"https:\/\/datascientest.com\/apache-kafka\">Kafka<\/a>, <a href=\"https:\/\/datascientest.com\/nlp-twitter-analyse-de-sentiment\">Twitter<\/a> ou <a href=\"https:\/\/datascientest.com\/apache-flume-tout-savoir\">Flume<\/a> ou en appliquant des op\u00e9rations de haut niveau sur d\u2019autres DStreams et peuvent \u00eatre utilis\u00e9s pour effectuer diverses op\u00e9rations comme le filtrage, l\u2019agr\u00e9gation, la jointure, etc., afin de produire les donn\u00e9es r\u00e9sultantes en temps r\u00e9el.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5306d6a elementor-widget elementor-widget-image\" data-id=\"5306d6a\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1290\" height=\"408\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image3-1.png\" class=\"attachment-full size-full wp-image-190418\" alt=\"\" loading=\"lazy\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image3-1.png 1290w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image3-1-300x95.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image3-1-1024x324.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image3-1-768x243.png 768w\" sizes=\"(max-width: 1290px) 100vw, 1290px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e582bdc elementor-widget elementor-widget-heading\" data-id=\"e582bdc\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\"><u>Prenons un exemple :<\/u><\/h4>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f304481 elementor-widget elementor-widget-text-editor\" data-id=\"f304481\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Imaginons que vous travaillez pour une entreprise de transport en commun qui souhaite surveiller en temps r\u00e9el les passagers qui montent et qui descendent des bus. Pour cela, les bus sont \u00e9quip\u00e9s de capteurs qui envoient des informations en continu aux serveurs de la soci\u00e9t\u00e9.\u00a0<\/span><\/p><p><span style=\"font-weight: 400;\">Spark Streaming nous permettra d\u2019utiliser un DStream pour traiter ces donn\u00e9es en temps r\u00e9el, o\u00f9 chaque DStream repr\u00e9sentera les donn\u00e9es envoy\u00e9es par les capteurs des bus dans des intervalles de temps pr\u00e9d\u00e9finis. Nous utiliserons des op\u00e9rations sur ces DStreams pour effectuer nos analyses en temps r\u00e9el. A l\u2019aide d\u2019agr\u00e9gations, nous pourrons obtenir le nombre total de passagers dans les bus \u00e0 chaque instant. Nous pourrons \u00e9galement utiliser des filtres afin d\u2019identifier les bus qui ont atteint leur capacit\u00e9 maximale et ainsi envoyer des alertes pour la s\u00e9curit\u00e9 des passagers.<\/span><\/p><p><span style=\"font-weight: 400;\">En somme, gr\u00e2ce \u00e0 Spark Streaming nous pouvons effectuer des analyses en temps r\u00e9el sur les donn\u00e9es envoy\u00e9es par les capteurs, et utiliser les r\u00e9sultats pour am\u00e9liorer la s\u00e9curit\u00e9 et l\u2019efficacit\u00e9 du syst\u00e8me de transport en commun.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-2e835af e-flex e-con-boxed e-con\" data-id=\"2e835af\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-696f74c elementor-widget elementor-widget-heading\" data-id=\"696f74c\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Qu'en est-il de la tol\u00e9rance de pannes et r\u00e9plication ?<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-24f316a elementor-widget elementor-widget-text-editor\" data-id=\"24f316a\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Spark Streaming garantit la <strong>tol\u00e9rance de panne<\/strong> gr\u00e2ce \u00e0 des techniques de r\u00e9plication de donn\u00e9es et de reprise de t\u00e2ches.<\/span><\/p><ul><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">La <strong>r\u00e9plication de donn\u00e9es<\/strong> consiste \u00e0 dupliquer les donn\u00e9es sur plusieurs n\u0153uds afin de garantir leur disponibilit\u00e9 en cas de panne d\u2019un n\u0153ud. Cette mesure assure \u00e9galement la robustesse du syst\u00e8me en garantissant que les donn\u00e9es ne seront pas perdues.<\/span><\/li><li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">La <strong>reprise de t\u00e2che<\/strong>, quant \u00e0 elle, est un m\u00e9canisme qui permet de red\u00e9marrer les t\u00e2ches sur d\u2019autres n\u0153uds en cas de panne, afin de garantir la continuit\u00e9 du traitement des donn\u00e9es m\u00eame en cas de probl\u00e8me.<\/span><\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6db5ede elementor-widget elementor-widget-image\" data-id=\"6db5ede\" data-element_type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1200\" height=\"628\" src=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image1-1.png\" class=\"attachment-full size-full wp-image-190419\" alt=\"\" loading=\"lazy\" srcset=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image1-1.png 1200w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image1-1-300x157.png 300w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image1-1-1024x536.png 1024w, https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/image1-1-768x402.png 768w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-5c8dde4 e-flex e-con-boxed e-con\" data-id=\"5c8dde4\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-42521d0 elementor-widget elementor-widget-heading\" data-id=\"42521d0\" data-element_type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Conclusion<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d28180b elementor-widget elementor-widget-text-editor\" data-id=\"d28180b\" data-element_type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<p><span style=\"font-weight: 400;\">Spark Streaming permet donc de r\u00e9pondre \u00e0 des probl\u00e9matiques de <a href=\"https:\/\/datascientest.com\/formation-big-data\">Big Data<\/a> en temps r\u00e9el. Le fait qu\u2019il supporte des sources de donn\u00e9es vari\u00e9es, et qu\u2019il permette de n\u2019utiliser qu\u2019un seul framework pour des besoins aussi vari\u00e9s n\u2019en demeure pas moins un avantage non n\u00e9gligeable.<\/span><\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8178fb7 elementor-align-center button-principale elementor-widget elementor-widget-button\" data-id=\"8178fb7\" data-element_type=\"widget\" data-widget_type=\"button.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-button-wrapper\">\n\t\t\t<a class=\"elementor-button elementor-button-link elementor-size-sm elementor-animation-grow\" href=\"https:\/\/datascientest.com\/formations-data\">\n\t\t\t\t\t\t<span class=\"elementor-button-content-wrapper\">\n\t\t\t\t\t\t<span class=\"elementor-button-text\">Commencer une formation sur le data streaming<\/span>\n\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div class=\"elementor-element elementor-element-7b5ba36 e-flex e-con-boxed e-con\" data-id=\"7b5ba36\" data-element_type=\"container\" data-settings=\"{&quot;content_width&quot;:&quot;boxed&quot;}\">\n\t\t\t\t\t<div class=\"e-con-inner\">\n\t\t\t\t<div class=\"elementor-element elementor-element-efadc67 elementor-widget elementor-widget-html\" data-id=\"efadc67\" data-element_type=\"widget\" data-widget_type=\"html.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t<script type=\"application\/ld+json\">\r\n{\r\n  \"@context\": \"https:\/\/schema.org\",\r\n  \"@type\": \"FAQPage\",\r\n  \"mainEntity\": [{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"Comment fonctionne le streaming de donn\u00e9es ?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"Spark Streaming utilise une architecture dite \u00ab micro-batch \u00bb, ce qui signifie que les donn\u00e9es sont divis\u00e9es en petits lots appel\u00e9s \u00ab batchs \u00bb et sont trait\u00e9es de mani\u00e8re s\u00e9quentielle. Chaque batch est trait\u00e9 comme un RDD (Resilient Distributed Dataset) dans Spark, ce qui permet de b\u00e9n\u00e9ficier de la puissance de traitement en parall\u00e8le de Spark. Pour rappel, un RDD est l\u2019unit\u00e9 de base de donn\u00e9es dans Apache Spark, qui est une collection immuable de donn\u00e9es partag\u00e9es parall\u00e8lement sur plusieurs n\u0153uds d\u2019un cluster.\"\r\n    }\r\n  },{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"Qu'est ce que les DStreams ?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"Spark Streaming fournit une abstraction de haut niveau appel\u00e9e \u00ab flux discr\u00e9tis\u00e9 \u00bb (Discretized Stream), ou DStream, qui repr\u00e9sentent un flux continu de donn\u00e9es. Ils peuvent \u00eatre consid\u00e9r\u00e9s comme des RDD continus, o\u00f9 chaque RDD repr\u00e9sente les donn\u00e9es g\u00e9n\u00e9r\u00e9es dans un intervalle d\u00e9fini.\"\r\n    }\r\n  },{\r\n    \"@type\": \"Question\",\r\n    \"name\": \"Qu'en est-il de la tol\u00e9rance de pannes et r\u00e9plication ?\",\r\n    \"acceptedAnswer\": {\r\n      \"@type\": \"Answer\",\r\n      \"text\": \"La r\u00e9plication de donn\u00e9es consiste \u00e0 dupliquer les donn\u00e9es sur plusieurs n\u0153uds afin de garantir leur disponibilit\u00e9 en cas de panne d\u2019un n\u0153ud. Cette mesure assure \u00e9galement la robustesse du syst\u00e8me en garantissant que les donn\u00e9es ne seront pas perdues.\"\r\n    }\r\n  }]\r\n}\r\n<\/script>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Spark Streaming est une solution innovante pour le traitement en temps r\u00e9el des donn\u00e9es. Il s\u2019agit d\u2019un composant du framework Apache Spark qui offre une performance, une scalabilit\u00e9 et une fiabilit\u00e9 exceptionnelles. Ce syst\u00e8me de traitement en temps r\u00e9el distribu\u00e9 a \u00e9t\u00e9 con\u00e7u pour r\u00e9pondre aux exigences les plus complexes en mati\u00e8re de traitement de [&hellip;]<\/p>\n","protected":false},"author":78,"featured_media":190415,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_theme","format":"standard","meta":{"content-type":"","inline_featured_image":false},"categories":[2497],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.7 (Yoast SEO v21.7) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Spark Streaming : Qu&#039;est-ce que c&#039;est ? Comment \u00e7a fonctionne ?<\/title>\n<meta name=\"description\" content=\"Les domaines d\u2019application d\u2019Apache Spark Streaming sont tr\u00e8s vari\u00e9s. Apprenez-en un peu plus \u00e0 ce sujet dans cet article.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Spark Streaming : Qu&#039;est-ce que c&#039;est ? Comment \u00e7a fonctionne ?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir\" \/>\n<meta property=\"og:site_name\" content=\"Formation Data Science | DataScientest.com\" \/>\n<meta property=\"article:published_time\" content=\"2023-02-20T17:59:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-11-09T13:54:02+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/spark-streaming.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"J\u00e9r\u00e9my Robert\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"J\u00e9r\u00e9my Robert\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir\",\"url\":\"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir\",\"name\":\"Spark Streaming : Qu'est-ce que c'est ? Comment \u00e7a fonctionne ?\",\"isPartOf\":{\"@id\":\"https:\/\/datascientest.com\/#website\"},\"datePublished\":\"2023-02-20T17:59:00+00:00\",\"dateModified\":\"2023-11-09T13:54:02+00:00\",\"author\":{\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/7af2361c48e24e79a8aa2360e6e74925\"},\"description\":\"Les domaines d\u2019application d\u2019Apache Spark Streaming sont tr\u00e8s vari\u00e9s. Apprenez-en un peu plus \u00e0 ce sujet dans cet article.\",\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir\"]}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/datascientest.com\/#website\",\"url\":\"https:\/\/datascientest.com\/\",\"name\":\"Formation Data Science | DataScientest.com\",\"description\":\"Leader de la formation en Data Science\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/datascientest.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/7af2361c48e24e79a8aa2360e6e74925\",\"name\":\"J\u00e9r\u00e9my Robert\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/datascientest.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/jeremyrobert.jpeg\",\"contentUrl\":\"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/jeremyrobert.jpeg\",\"caption\":\"J\u00e9r\u00e9my Robert\"},\"description\":\"J\u00e9r\u00e9my Robert est actuellement Directeur p\u00e9dagogique B2B chez DataScientest.com. Avant de rejoindre DataScientest, il a travaill\u00e9 en tant que Consultant Business Intelligence chez DEAPDATA, o\u00f9 il a g\u00e9r\u00e9 des projets dans les domaines de l'assurance, bancaire et industrie. Il a \u00e9galement \u00e9t\u00e9 Co-fondateur et pr\u00e9sident de Tripzer. En tant que consultant ind\u00e9pendant, J\u00e9r\u00e9my a propos\u00e9 des formations sur MicroStrategy pour divers niveaux professionnels. Il est dipl\u00f4m\u00e9 de l'Universit\u00e9 Paris Cit\u00e9 avec un Master MIAGE, combinant comp\u00e9tences en informatique et gestion.\",\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/jeremyrobert1\/\",\"Data Science\"],\"url\":\"https:\/\/datascientest.com\/author\/jeremyrobert\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Spark Streaming : Qu'est-ce que c'est ? Comment \u00e7a fonctionne ?","description":"Les domaines d\u2019application d\u2019Apache Spark Streaming sont tr\u00e8s vari\u00e9s. Apprenez-en un peu plus \u00e0 ce sujet dans cet article.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir","og_locale":"fr_FR","og_type":"article","og_title":"Spark Streaming : Qu'est-ce que c'est ? Comment \u00e7a fonctionne ?","og_url":"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir","og_site_name":"Formation Data Science | DataScientest.com","article_published_time":"2023-02-20T17:59:00+00:00","article_modified_time":"2023-11-09T13:54:02+00:00","og_image":[{"width":1200,"height":600,"url":"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/02\/spark-streaming.png","type":"image\/png"}],"author":"J\u00e9r\u00e9my Robert","twitter_card":"summary_large_image","twitter_misc":{"\u00c9crit par":"J\u00e9r\u00e9my Robert","Dur\u00e9e de lecture estim\u00e9e":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir","url":"https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir","name":"Spark Streaming : Qu'est-ce que c'est ? Comment \u00e7a fonctionne ?","isPartOf":{"@id":"https:\/\/datascientest.com\/#website"},"datePublished":"2023-02-20T17:59:00+00:00","dateModified":"2023-11-09T13:54:02+00:00","author":{"@id":"https:\/\/datascientest.com\/#\/schema\/person\/7af2361c48e24e79a8aa2360e6e74925"},"description":"Les domaines d\u2019application d\u2019Apache Spark Streaming sont tr\u00e8s vari\u00e9s. Apprenez-en un peu plus \u00e0 ce sujet dans cet article.","inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/datascientest.com\/apache-spark-streaming-tout-savoir"]}]},{"@type":"WebSite","@id":"https:\/\/datascientest.com\/#website","url":"https:\/\/datascientest.com\/","name":"Formation Data Science | DataScientest.com","description":"Leader de la formation en Data Science","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/datascientest.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"fr-FR"},{"@type":"Person","@id":"https:\/\/datascientest.com\/#\/schema\/person\/7af2361c48e24e79a8aa2360e6e74925","name":"J\u00e9r\u00e9my Robert","image":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/datascientest.com\/#\/schema\/person\/image\/","url":"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/jeremyrobert.jpeg","contentUrl":"https:\/\/datascientest.com\/wp-content\/uploads\/2023\/11\/jeremyrobert.jpeg","caption":"J\u00e9r\u00e9my Robert"},"description":"J\u00e9r\u00e9my Robert est actuellement Directeur p\u00e9dagogique B2B chez DataScientest.com. Avant de rejoindre DataScientest, il a travaill\u00e9 en tant que Consultant Business Intelligence chez DEAPDATA, o\u00f9 il a g\u00e9r\u00e9 des projets dans les domaines de l'assurance, bancaire et industrie. Il a \u00e9galement \u00e9t\u00e9 Co-fondateur et pr\u00e9sident de Tripzer. En tant que consultant ind\u00e9pendant, J\u00e9r\u00e9my a propos\u00e9 des formations sur MicroStrategy pour divers niveaux professionnels. Il est dipl\u00f4m\u00e9 de l'Universit\u00e9 Paris Cit\u00e9 avec un Master MIAGE, combinant comp\u00e9tences en informatique et gestion.","sameAs":["https:\/\/www.linkedin.com\/in\/jeremyrobert1\/","Data Science"],"url":"https:\/\/datascientest.com\/author\/jeremyrobert"}]}},"_links":{"self":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts\/190414"}],"collection":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/users\/78"}],"replies":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/comments?post=190414"}],"version-history":[{"count":0,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/posts\/190414\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/media\/190415"}],"wp:attachment":[{"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/media?parent=190414"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/categories?post=190414"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/datascientest.com\/wp-json\/wp\/v2\/tags?post=190414"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}