{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "MNIST_better_solution.ipynb", "provenance": [], "collapsed_sections": [] }, "kernelspec": { "name": "python3", "display_name": "Python 3" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "1SX5oNyuYbhq", "colab_type": "text" }, "source": [ "# Lösung zum MNIST Datensatz mit 99.25% Genauigkeit " ] }, { "cell_type": "markdown", "metadata": { "id": "b0LtRcCeYt5v", "colab_type": "text" }, "source": [ "**HINWEIS:** Keine Angst, ihr müsst nicht verstehen was/wie in diesem Notebook gemacht wird. Wir werden euch noch die kommenden Wochen genauer erklären, was die einzelnen Netzeigenschaften sind und wofür sie gut sind. \n", "Wir wollten euch diese Lösung nur mitgeben, um zu zeigen, dass wir auch eine weitaus bessere Genauigkeit, als in dem Video zu der MNIST Lösung in Woche 2, erreichen können.\n", "\n", "Seht es somit als Motivation, was wir in dem Kurs alles noch lernen und wie stark wir uns damit verbessern werden." ] }, { "cell_type": "markdown", "metadata": { "id": "PwgcVJf2XqiJ", "colab_type": "text" }, "source": [ "In diesem Notebook wird ein \"Convolutional Neural Network\" (was Convolutional bedeutet erfahrt ihr in Woche 3) mit dem MNIST Datensatz verwendet. \n", "\n", "Dieses Netz erreicht bis zu 99.25% Genauigkeit bei den Testdaten nach 12 Epochen. Hierbei kann eventuell noch eine höhere Genauigkeit erreicht werden, wenn man an den Parametern herum schraubt. Solltet ihr daran interesse haben und generell sehen wollen, wie Veränderungen der Parameter eventuell die Genauigkeit beeinflussen, dann spielt damit doch einmal herum. " ] }, { "cell_type": "code", "metadata": { "id": "lIBLDTxm7qDA", "colab_type": "code", "outputId": "015e7264-22d4-4007-efe4-30fa9cff71ba", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 } }, "source": [ "from __future__ import print_function\n", "import keras\n", "from keras.datasets import mnist\n", "from keras.models import Sequential\n", "from keras.layers import Dense # Diese Art von Schicht kennen wir bereits\n", "from keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten # Diese Schichten lernen wir in Woche 3 und 4 näher kennen\n", "from keras import backend as K\n", "\n", "batch_size = 128 # Batch Größe\n", "num_classes = 10 # Anzahl der Klassen (Ausgabeneuronen - Zahlen zwischen 0 und 9)\n", "epochs = 12 # Anzahl der Epochen\n", "img_rows, img_cols = 28, 28 # Dimension der Eingabedaten\n", "\n", "(x_train, y_train), (x_test, y_test) = mnist.load_data() # Laden der Daten und aufteilen in Trainings- und Testdaten\n", "\n", "# Notwendige Umformungen der Daten für die anschließende Verwendung in dem \"Convolutional Neural Network\"\n", "if K.image_data_format() == 'channels_first':\n", " x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)\n", " x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)\n", " input_shape = (1, img_rows, img_cols)\n", "else:\n", " x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n", " x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n", " input_shape = (img_rows, img_cols, 1)\n", "\n", "x_train = x_train.astype('float32') # notwendige Datentypkonvertierung\n", "x_test = x_test.astype('float32') # notwendige Datentypkonvertierung\n", "x_train /= 255 # Normalisierung der Trainingsdaten\n", "x_test /= 255 # Normalisierung der Testdaten\n", "\n", "# Wie bereits bekannt, erwarten wir die Labels als Vektor (z.b. 5 als [0,0,0,0,0,1,0,0,0,0])\n", "y_train = keras.utils.to_categorical(y_train, num_classes) # Umformung der Trainingslabel\n", "y_test = keras.utils.to_categorical(y_test, num_classes) # Umformung der Testlabel\n", "\n", "model = Sequential([\n", " Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=input_shape), # Werden wir in Woche 3 kennenlernen!\n", " Conv2D(64, (3, 3), activation='relu'), # Werden wir in Woche 3 kennenlernen!\n", " MaxPooling2D(pool_size=(2, 2)), # Werden wir in Woche 3 kennenlernen!\n", " Dropout(0.25), # Werden wir in Woche 4 kennenlernen!\n", " Flatten(), # Werden wir in Woche 3 kennenlernen!\n", " Dense(128, activation='relu'), # Kennen wir bereits!\n", " Dropout(0.5), # Werden wir in Woche 4 kennenlernen!\n", " Dense(num_classes, activation='softmax'), # Die Aktivierungsfunktion Softmax werden wir in Woche 3 kennenlernen!\n", "])\n", "\n", "model.compile(loss=keras.losses.categorical_crossentropy, # Die Kostenfunktion categorical_crossentropy werden wir in Woche 3 kennenlernen!\n", " optimizer=keras.optimizers.Adadelta(), # Den Optimizer Adadelta werden wir in Woche 3 kennenlernen!\n", " metrics=['accuracy'])\n", "\n", "model.fit(x_train, y_train,\n", " batch_size=batch_size, # Kennen wir bereits!\n", " epochs=epochs, # Kennen wir bereits!\n", " verbose=1,\n", " validation_data=(x_test, y_test)) # Wir validieren nach jeder Epoche, wie hoch die derzeitige Genauigkeit ist und übergeben hierfür die gesamten Testdaten (mit diesen wird nicht trainiert).\n", "\n", "score = model.evaluate(x_test, y_test, verbose=0) # Wir berechnen die Genauigkeit des Netzes, um sie nachfolgend ordentlich anzugeben\n", "\n", "print('Fehler auf den Testdaten:', score[0]) # Ausgabe der Ergebnisse\n", "print('Accuracy auf den Testdaten:', score[1])" ], "execution_count": 0, "outputs": [ { "output_type": "stream", "text": [ "Using TensorFlow backend.\n" ], "name": "stderr" }, { "output_type": "display_data", "data": { "text/html": [ "

\n", "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.
\n", "We recommend you upgrade now \n", "or ensure your notebook will continue to use TensorFlow 1.x via the %tensorflow_version 1.x magic:\n", "more info.

\n" ], "text/plain": [ "" ] }, "metadata": { "tags": [] } }, { "output_type": "stream", "text": [ "Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz\n", "11493376/11490434 [==============================] - 0s 0us/step\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4267: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:148: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3733: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3576: The name tf.log is deprecated. Please use tf.math.log instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", "Instructions for updating:\n", "Use tf.where in 2.0, which has the same broadcast rule as np.where\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1033: The name tf.assign_add is deprecated. Please use tf.compat.v1.assign_add instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1020: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3005: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n", "\n", "Train on 60000 samples, validate on 10000 samples\n", "Epoch 1/12\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:197: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:207: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:216: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.\n", "\n", "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:223: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.\n", "\n", "60000/60000 [==============================] - 151s 3ms/step - loss: 0.2701 - acc: 0.9168 - val_loss: 0.0646 - val_acc: 0.9796\n", "Epoch 2/12\n", "60000/60000 [==============================] - 150s 2ms/step - loss: 0.0893 - acc: 0.9740 - val_loss: 0.0395 - val_acc: 0.9869\n", "Epoch 3/12\n", "60000/60000 [==============================] - 150s 3ms/step - loss: 0.0669 - acc: 0.9803 - val_loss: 0.0383 - val_acc: 0.9875\n", "Epoch 4/12\n", "60000/60000 [==============================] - 150s 3ms/step - loss: 0.0534 - acc: 0.9842 - val_loss: 0.0352 - val_acc: 0.9891\n", "Epoch 5/12\n", "60000/60000 [==============================] - 150s 2ms/step - loss: 0.0468 - acc: 0.9863 - val_loss: 0.0280 - val_acc: 0.9902\n", "Epoch 6/12\n", "60000/60000 [==============================] - 150s 3ms/step - loss: 0.0415 - acc: 0.9880 - val_loss: 0.0272 - val_acc: 0.9912\n", "Epoch 7/12\n", "60000/60000 [==============================] - 149s 2ms/step - loss: 0.0380 - acc: 0.9889 - val_loss: 0.0320 - val_acc: 0.9883\n", "Epoch 8/12\n", "60000/60000 [==============================] - 150s 2ms/step - loss: 0.0341 - acc: 0.9897 - val_loss: 0.0293 - val_acc: 0.9909\n", "Epoch 9/12\n", "60000/60000 [==============================] - 150s 2ms/step - loss: 0.0296 - acc: 0.9909 - val_loss: 0.0255 - val_acc: 0.9912\n", "Epoch 10/12\n", "60000/60000 [==============================] - 150s 2ms/step - loss: 0.0291 - acc: 0.9910 - val_loss: 0.0280 - val_acc: 0.9912\n", "Epoch 11/12\n", "60000/60000 [==============================] - 149s 2ms/step - loss: 0.0257 - acc: 0.9921 - val_loss: 0.0346 - val_acc: 0.9900\n", "Epoch 12/12\n", "60000/60000 [==============================] - 149s 2ms/step - loss: 0.0257 - acc: 0.9919 - val_loss: 0.0281 - val_acc: 0.9924\n", "Fehler auf den Testdaten: 0.02806703284695959\n", "Accuracy auf den Testdaten: 0.9924\n" ], "name": "stdout" } ] } ] }