303 lines
8.9 KiB
Plaintext
303 lines
8.9 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"# Exercice de traitement de données avec Pandas\n",
|
||
"\n",
|
||
"Tiré du site de Guillaume Dueymes : https://www.guillaumedueymes.com/courses/formation_python/8-pandas-exercice/\n",
|
||
"\n",
|
||
"Nous allons analyser un data set contenant des informations sur 10 000 paires de chaussures \n",
|
||
"vendues sur le site Amazon, avec de nombreuses caractéristiques comme le prix minimal observé, \n",
|
||
"le prix maximal observé, les couleurs disponibles, les tailles disponibles, le poids des articles, \n",
|
||
"la marque…\n"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "ca962ad386449748"
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"## Découverte du data set\n",
|
||
"\n",
|
||
"1. À l’aide de la fonction `read_csv()`, importez entièrement le data set et enregistrez-le dans une variable `shoes`.\n",
|
||
"2. Utilisez la méthode `.head()` pour afficher les premières lignes du `DataFrame`.\n",
|
||
"3. Il y a plus de 4 colonnes, beaucoup ne sont pas visibles. Afin de toutes les voir lors de l'affichage, utilisez la fonction `pandas.set_option()` pour que `.head()` affiche toutes les colonnes du `DataFrame`. (consultez la documentation de `set_option()`)\n",
|
||
"4. On va garder uniquement les colonnes intéressantes. Grâce à la syntaxe de filtrage par colonnes, créez une variable `shoes_light`, comprenant uniquement les colonnes suivantes : `id`, `name`, `brand`, dateUpdated`, `colors`, `prices.amountMax`, `prices.amountMin` et `prices.merchant`. Affichez le `head()` de `shoes_light`.\n"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "2f0fb688fd2b801e"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Requirement already satisfied: pandas in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (2.1.1)\r\n",
|
||
"Requirement already satisfied: numpy>=1.23.2 in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (from pandas) (1.26.0)\r\n",
|
||
"Requirement already satisfied: python-dateutil>=2.8.2 in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (from pandas) (2.8.2)\r\n",
|
||
"Requirement already satisfied: pytz>=2020.1 in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (from pandas) (2023.3.post1)\r\n",
|
||
"Requirement already satisfied: tzdata>=2022.1 in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (from pandas) (2023.3)\r\n",
|
||
"Requirement already satisfied: six>=1.5 in /home/steve/Code/python/.venv/datascience/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)\r\n",
|
||
"\r\n",
|
||
"\u001B[1m[\u001B[0m\u001B[34;49mnotice\u001B[0m\u001B[1;39;49m]\u001B[0m\u001B[39;49m A new release of pip is available: \u001B[0m\u001B[31;49m23.3.1\u001B[0m\u001B[39;49m -> \u001B[0m\u001B[32;49m23.3.2\u001B[0m\r\n",
|
||
"\u001B[1m[\u001B[0m\u001B[34;49mnotice\u001B[0m\u001B[1;39;49m]\u001B[0m\u001B[39;49m To update, run: \u001B[0m\u001B[32;49mpip install --upgrade pip\u001B[0m\r\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"!pip install pandas\n",
|
||
"import pandas as pd # noqa"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false,
|
||
"ExecuteTime": {
|
||
"end_time": "2023-12-19T09:59:24.292192547Z",
|
||
"start_time": "2023-12-19T09:59:21.699160182Z"
|
||
}
|
||
},
|
||
"id": "fb6b33145b46036b"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 1\n",
|
||
"shoes = pd.read_csv(\"womens-shoes.csv.xz\")"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "1b86b0cd93c36795"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 2"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "2efcdad9ec4b7583"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 3"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "2961d6e42459dfa9"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 4"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "6ee0a6019e90db1b"
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"## Data Cleaning\n",
|
||
"\n",
|
||
"1. À l'aide de l'attribut `.dtypes` du dataframe, regardez attentivement les types de chaque colonne. Certaines ont un type qui n'est pas celui attendu. Lesquelles ?\n",
|
||
"2. À l'aide des méthodes `.isnull()` (ou `.isna()`), `.sum()` et `len()`, calculez pour chaque colonne le pourcentage de valeurs non renseignées. Notez quelque part celles qui ont un non remplissage supérieur à 10%. La méthode `sum()` employée sur une série de booléens fait l'addition en considérant que `False == 0` et `True == 1`.\n",
|
||
"3. Supprimez du dataframe `shoes_light` les colonnes que vous avez notées dans la question précédente, elles ont trop de valeurs non renseignées.\n",
|
||
"4. À l'aide de la méthode `.to_datetime()` du dataframe, convertissez le type de la colonne `dateUpdated`.\n"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "865055e156e3a14c"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 1"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "5856f3288e1b837c"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 2"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "8b3be0e49e5edd4e"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 3"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "3d11033148a17412"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 4"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "6bc78f0eab8a090e"
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"## Features Modeling\n",
|
||
"\n",
|
||
"1. Ajoutez au dataframe une nouvelle colonne `prices.amountAverage` calculant la moyenne des colonnes `prices.amountMax` et `prices.amountMin` (via une addition et une division par 2).\n",
|
||
"2. Grâce à l'attribut `Series.dt.weekday`, ajoutez au dataframe une nouvelle colonne `dayOfweekUpdated`, extrayant depuis la colonne `dateUpdated` le jour de la semaine où les produits sont mis à jour (un nombre entre 0 et 6).\n"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "97a23a2b56e659b3"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 1"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "c1c2d9b73cf09b13"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 2"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "f79a989bd03e4f61"
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"source": [
|
||
"## Data Analyse\n",
|
||
"\n",
|
||
"1. Affichez le prix moyen, écart type, etc. des chaussures avec la méthode `.describe()`.\n",
|
||
"2. Y a-t-il de grandes différences de prix en fonction de la marque ? À l'aide des méthodes `groupby()`, `mean()` et `sort_values()`, créez une variable `luxury` contenant les 10 marques les plus chères, puis une variable `low_cost` contenant les 10 marques les moins chères.\n",
|
||
"3. Grâce à la méthode `value_counts()`, déterminez le jour de la semaine où les produits sont le plus souvent mis à jour.\n",
|
||
"4. **(Optionnel)** Donnez le prix moyen des produits de la marque `easy street` mis à jour un jeudi (jour 3).\n"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "1980ec43bdbabe8"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 1"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "73df484243c117c5"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 2"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "7f1abc9d9f255936"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 3"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "bae81f6190201b34"
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"outputs": [],
|
||
"source": [
|
||
"# Question 4"
|
||
],
|
||
"metadata": {
|
||
"collapsed": false
|
||
},
|
||
"id": "f46f052425a9711f"
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 2
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython2",
|
||
"version": "2.7.6"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|