TechTogetWorld

인공지능 구현에 대한 글입니다.


글의 순서는 아래와 같습니다.


================================================

1.# lab-11-1-mnist_cnn

    ==> http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html : cnn 시물레이션

    ==> 2단 CNN => 정확도 98.83%

    ==> CNN은 수행시간이 많이 소요됨, 코드 테스트시에는 TRAINING EPOCHS를 줄여 실행할 필요가 있음.

         물론 구글, 아마존 크라우드 서비스 활용도 가능함( 경우에 따라 비용발생)

         그래픽 카드 (GPU NVIDIA )를 적용하는 방법도 있음.


2. # lab-11-2-mnist_deep_cnn 

   ==> 3단 CNN : 정확도 99.38 %


3. # lab-11-3-mnist_cnn_class

   ==> python의 clss를 이용해서 코드의 반복을 줄이고, 코드를 단순화함


4. #lab-11-4-mnist_cnn_layers

   ==> tf.layers 패키지 사용하면 코드가 더욱 단순화 될수 있음

   ==> https://www.tensorflow.org/api_docs/python/tf/layers

       

5. # lab-11-5-mnist_cnn_ensemble_layers

  ==> 각각 예측시키고, 후에 병합을 해서 예측치를 출력시킴

  ==> 정확도 : 99.52%


6. 코드탐구(추가)

  ==>lab-11-X-mnist_cnn_low_memory


7. 참고자료

=================================================


[  lab-11-1-mnist_cnn ]


# lab-11-1-mnist_cnn

"""http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html : cnn 시물레이션

   2단 CNN => 정확도 98.83%

"""

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 11 MNIST and Convolutional Neural Network

import tensorflow as tf

import random

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# hyper parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100


# input place holders

X = tf.placeholder(tf.float32, [None, 784])

X_img = tf.reshape(X, [-1, 28, 28, 1])   # img 28x28x1 (black/white)

Y = tf.placeholder(tf.float32, [None, 10])


# L1 ImgIn shape=(?, 28, 28, 1)

W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))

#    Conv     -> (?, 28, 28, 32)

#    Pool     -> (?, 14, 14, 32)

L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')

L1 = tf.nn.relu(L1)

L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],

                    strides=[1, 2, 2, 1], padding='SAME')

'''

Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)

Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)

Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)

'''


# L2 ImgIn shape=(?, 14, 14, 32)

W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))

#    Conv      ->(?, 14, 14, 64)

#    Pool      ->(?, 7, 7, 64)

L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')

L2 = tf.nn.relu(L2)

L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],

                    strides=[1, 2, 2, 1], padding='SAME')

L2_flat = tf.reshape(L2, [-1, 7 * 7 * 64])

'''

Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)

Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)

Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)

Tensor("Reshape_1:0", shape=(?, 3136), dtype=float32)

'''


# Final FC 7x7x64 inputs -> 10 outputs

W3 = tf.get_variable("W3", shape=[7 * 7 * 64, 10],

                     initializer=tf.contrib.layers.xavier_initializer())

b = tf.Variable(tf.random_normal([10]))

logits = tf.matmul(L2_flat, W3) + b


# define cost/loss & optimizer

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

    logits=logits, labels=Y))

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)


# initialize

sess = tf.Session()

sess.run(tf.global_variables_initializer())


# train my model

print('Learning started. It takes sometime.')

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        feed_dict = {X: batch_xs, Y: batch_ys}

        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print('Accuracy:', sess.run(accuracy, feed_dict={

      X: mnist.test.images, Y: mnist.test.labels}))


# Get one and predict

r = random.randint(0, mnist.test.num_examples - 1)

print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

print("Prediction: ", sess.run(

    tf.argmax(logits, 1), feed_dict={X: mnist.test.images[r:r + 1]}))


# plt.imshow(mnist.test.images[r:r + 1].

#           reshape(28, 28), cmap='Greys', interpolation='nearest')

# plt.show()


'''

Epoch: 0001 cost = 0.340291267

Epoch: 0002 cost = 0.090731326

Epoch: 0003 cost = 0.064477619

Epoch: 0004 cost = 0.050683064

Epoch: 0005 cost = 0.041864835

Epoch: 0006 cost = 0.035760704

Epoch: 0007 cost = 0.030572132

Epoch: 0008 cost = 0.026207981

Epoch: 0009 cost = 0.022622454

Epoch: 0010 cost = 0.019055919

Epoch: 0011 cost = 0.017758641

Epoch: 0012 cost = 0.014156652

Epoch: 0013 cost = 0.012397016

Epoch: 0014 cost = 0.010693789

Epoch: 0015 cost = 0.009469977

Learning Finished!

Accuracy: 0.9885

'''



[ # lab-11-2-mnist_deep_cnn ]


# lab-11-2-mnist_deep_cnn

"""

3단 CNN : 정확도 99.38 %


"""

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 11 MNIST and Deep learning CNN

import tensorflow as tf

import random

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# hyper parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100


# dropout (keep_prob) rate  0.7~0.5 on training, but should be 1 for testing

keep_prob = tf.placeholder(tf.float32)


# input place holders

X = tf.placeholder(tf.float32, [None, 784])

X_img = tf.reshape(X, [-1, 28, 28, 1])   # img 28x28x1 (black/white)

Y = tf.placeholder(tf.float32, [None, 10])


# L1 ImgIn shape=(?, 28, 28, 1)

W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))

#    Conv     -> (?, 28, 28, 32)

#    Pool     -> (?, 14, 14, 32)

L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')

L1 = tf.nn.relu(L1)

L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],

                    strides=[1, 2, 2, 1], padding='SAME')

L1 = tf.nn.dropout(L1, keep_prob=keep_prob)

'''

Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)

Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)

Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)

Tensor("dropout/mul:0", shape=(?, 14, 14, 32), dtype=float32)

'''


# L2 ImgIn shape=(?, 14, 14, 32)

W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))

#    Conv      ->(?, 14, 14, 64)

#    Pool      ->(?, 7, 7, 64)

L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')

L2 = tf.nn.relu(L2)

L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],

                    strides=[1, 2, 2, 1], padding='SAME')

L2 = tf.nn.dropout(L2, keep_prob=keep_prob)

'''

Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)

Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)

Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)

Tensor("dropout_1/mul:0", shape=(?, 7, 7, 64), dtype=float32)

'''


# L3 ImgIn shape=(?, 7, 7, 64)

W3 = tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.01))

#    Conv      ->(?, 7, 7, 128)

#    Pool      ->(?, 4, 4, 128)

#    Reshape   ->(?, 4 * 4 * 128) # Flatten them for FC

L3 = tf.nn.conv2d(L2, W3, strides=[1, 1, 1, 1], padding='SAME')

L3 = tf.nn.relu(L3)

L3 = tf.nn.max_pool(L3, ksize=[1, 2, 2, 1], strides=[

                    1, 2, 2, 1], padding='SAME')

L3 = tf.nn.dropout(L3, keep_prob=keep_prob)

L3_flat = tf.reshape(L3, [-1, 128 * 4 * 4])

'''

Tensor("Conv2D_2:0", shape=(?, 7, 7, 128), dtype=float32)

Tensor("Relu_2:0", shape=(?, 7, 7, 128), dtype=float32)

Tensor("MaxPool_2:0", shape=(?, 4, 4, 128), dtype=float32)

Tensor("dropout_2/mul:0", shape=(?, 4, 4, 128), dtype=float32)

Tensor("Reshape_1:0", shape=(?, 2048), dtype=float32)

'''


# L4 FC 4x4x128 inputs -> 625 outputs

W4 = tf.get_variable("W4", shape=[128 * 4 * 4, 625],

                     initializer=tf.contrib.layers.xavier_initializer())

b4 = tf.Variable(tf.random_normal([625]))

L4 = tf.nn.relu(tf.matmul(L3_flat, W4) + b4)

L4 = tf.nn.dropout(L4, keep_prob=keep_prob)

'''

Tensor("Relu_3:0", shape=(?, 625), dtype=float32)

Tensor("dropout_3/mul:0", shape=(?, 625), dtype=float32)

'''


# L5 Final FC 625 inputs -> 10 outputs

W5 = tf.get_variable("W5", shape=[625, 10],

                     initializer=tf.contrib.layers.xavier_initializer())

b5 = tf.Variable(tf.random_normal([10]))

logits = tf.matmul(L4, W5) + b5

'''

Tensor("add_1:0", shape=(?, 10), dtype=float32)

'''


# define cost/loss & optimizer

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

    logits=logits, labels=Y))

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)


# initialize

sess = tf.Session()

sess.run(tf.global_variables_initializer())


# train my model

print('Learning started. It takes sometime.')

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        feed_dict = {X: batch_xs, Y: batch_ys, keep_prob: 0.7}

        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy


# if you have a OOM error, please refer to lab-11-X-mnist_deep_cnn_low_memory.py


correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print('Accuracy:', sess.run(accuracy, feed_dict={

      X: mnist.test.images, Y: mnist.test.labels, keep_prob: 1}))


# Get one and predict

r = random.randint(0, mnist.test.num_examples - 1)

print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

print("Prediction: ", sess.run(

    tf.argmax(logits, 1), feed_dict={X: mnist.test.images[r:r + 1], keep_prob: 1}))


# plt.imshow(mnist.test.images[r:r + 1].

#           reshape(28, 28), cmap='Greys', interpolation='nearest')

# plt.show()


'''

Learning stared. It takes sometime.

Epoch: 0001 cost = 0.385748474

Epoch: 0002 cost = 0.092017397

Epoch: 0003 cost = 0.065854684

Epoch: 0004 cost = 0.055604566

Epoch: 0005 cost = 0.045996377

Epoch: 0006 cost = 0.040913645

Epoch: 0007 cost = 0.036924479

Epoch: 0008 cost = 0.032808939

Epoch: 0009 cost = 0.031791007

Epoch: 0010 cost = 0.030224456

Epoch: 0011 cost = 0.026849916

Epoch: 0012 cost = 0.026826763

Epoch: 0013 cost = 0.027188021

Epoch: 0014 cost = 0.023604777

Epoch: 0015 cost = 0.024607201

Learning Finished!

Accuracy: 0.9938




[ # lab-11-3-mnist_cnn_class ]


# lab-11-3-mnist_cnn_class

""" python의 clss를 이용해서 코드의 반복을 줄이고, 코드를 단순화함"""


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 11 MNIST and Deep learning CNN

import tensorflow as tf

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# hyper parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100



class Model:


    def __init__(self, sess, name):

        self.sess = sess

        self.name = name

        self._build_net()


    def _build_net(self):

        with tf.variable_scope(self.name):

            # dropout (keep_prob) rate  0.7~0.5 on training, but should be 1

            # for testing

            self.keep_prob = tf.placeholder(tf.float32)


            # input place holders

            self.X = tf.placeholder(tf.float32, [None, 784])

            # img 28x28x1 (black/white)

            X_img = tf.reshape(self.X, [-1, 28, 28, 1])

            self.Y = tf.placeholder(tf.float32, [None, 10])


            # L1 ImgIn shape=(?, 28, 28, 1)

            W1 = tf.Variable(tf.random_normal([3, 3, 1, 32], stddev=0.01))

            #    Conv     -> (?, 28, 28, 32)

            #    Pool     -> (?, 14, 14, 32)

            L1 = tf.nn.conv2d(X_img, W1, strides=[1, 1, 1, 1], padding='SAME')

            L1 = tf.nn.relu(L1)

            L1 = tf.nn.max_pool(L1, ksize=[1, 2, 2, 1],

                                strides=[1, 2, 2, 1], padding='SAME')

            L1 = tf.nn.dropout(L1, keep_prob=self.keep_prob)

            '''

            Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)

            Tensor("Relu:0", shape=(?, 28, 28, 32), dtype=float32)

            Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)

            Tensor("dropout/mul:0", shape=(?, 14, 14, 32), dtype=float32)

            '''


            # L2 ImgIn shape=(?, 14, 14, 32)

            W2 = tf.Variable(tf.random_normal([3, 3, 32, 64], stddev=0.01))

            #    Conv      ->(?, 14, 14, 64)

            #    Pool      ->(?, 7, 7, 64)

            L2 = tf.nn.conv2d(L1, W2, strides=[1, 1, 1, 1], padding='SAME')

            L2 = tf.nn.relu(L2)

            L2 = tf.nn.max_pool(L2, ksize=[1, 2, 2, 1],

                                strides=[1, 2, 2, 1], padding='SAME')

            L2 = tf.nn.dropout(L2, keep_prob=self.keep_prob)

            '''

            Tensor("Conv2D_1:0", shape=(?, 14, 14, 64), dtype=float32)

            Tensor("Relu_1:0", shape=(?, 14, 14, 64), dtype=float32)

            Tensor("MaxPool_1:0", shape=(?, 7, 7, 64), dtype=float32)

            Tensor("dropout_1/mul:0", shape=(?, 7, 7, 64), dtype=float32)

            '''


            # L3 ImgIn shape=(?, 7, 7, 64)

            W3 = tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.01))

            #    Conv      ->(?, 7, 7, 128)

            #    Pool      ->(?, 4, 4, 128)

            #    Reshape   ->(?, 4 * 4 * 128) # Flatten them for FC

            L3 = tf.nn.conv2d(L2, W3, strides=[1, 1, 1, 1], padding='SAME')

            L3 = tf.nn.relu(L3)

            L3 = tf.nn.max_pool(L3, ksize=[1, 2, 2, 1], strides=[

                                1, 2, 2, 1], padding='SAME')

            L3 = tf.nn.dropout(L3, keep_prob=self.keep_prob)


            L3_flat = tf.reshape(L3, [-1, 128 * 4 * 4])

            '''

            Tensor("Conv2D_2:0", shape=(?, 7, 7, 128), dtype=float32)

            Tensor("Relu_2:0", shape=(?, 7, 7, 128), dtype=float32)

            Tensor("MaxPool_2:0", shape=(?, 4, 4, 128), dtype=float32)

            Tensor("dropout_2/mul:0", shape=(?, 4, 4, 128), dtype=float32)

            Tensor("Reshape_1:0", shape=(?, 2048), dtype=float32)

            '''


            # L4 FC 4x4x128 inputs -> 625 outputs

            W4 = tf.get_variable("W4", shape=[128 * 4 * 4, 625],

                                 initializer=tf.contrib.layers.xavier_initializer())

            b4 = tf.Variable(tf.random_normal([625]))

            L4 = tf.nn.relu(tf.matmul(L3_flat, W4) + b4)

            L4 = tf.nn.dropout(L4, keep_prob=self.keep_prob)

            '''

            Tensor("Relu_3:0", shape=(?, 625), dtype=float32)

            Tensor("dropout_3/mul:0", shape=(?, 625), dtype=float32)

            '''


            # L5 Final FC 625 inputs -> 10 outputs

            W5 = tf.get_variable("W5", shape=[625, 10],

                                 initializer=tf.contrib.layers.xavier_initializer())

            b5 = tf.Variable(tf.random_normal([10]))

            self.logits = tf.matmul(L4, W5) + b5

            '''

            Tensor("add_1:0", shape=(?, 10), dtype=float32)

            '''


        # define cost/loss & optimizer

        self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

            logits=self.logits, labels=self.Y))

        self.optimizer = tf.train.AdamOptimizer(

            learning_rate=learning_rate).minimize(self.cost)


        correct_prediction = tf.equal(

            tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))

        self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


    def predict(self, x_test, keep_prop=1.0):

        return self.sess.run(self.logits, feed_dict={self.X: x_test, self.keep_prob: keep_prop})


    def get_accuracy(self, x_test, y_test, keep_prop=1.0):

        return self.sess.run(self.accuracy, feed_dict={self.X: x_test, self.Y: y_test, self.keep_prob: keep_prop})


    def train(self, x_data, y_data, keep_prop=0.7):

        return self.sess.run([self.cost, self.optimizer], feed_dict={

            self.X: x_data, self.Y: y_data, self.keep_prob: keep_prop})


# initialize

sess = tf.Session()

m1 = Model(sess, "m1")


sess.run(tf.global_variables_initializer())


print('Learning Started!')


# train my model

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        c, _ = m1.train(batch_xs, batch_ys)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

print('Accuracy:', m1.get_accuracy(mnist.test.images, mnist.test.labels))



[#lab-11-4-mnist_cnn_layers]


#lab-11-4-mnist_cnn_layers

"""

tf.layers 패키지 사용하면 코드가 더욱 단순화 될수 있음

 https://www.tensorflow.org/api_docs/python/tf/layers

CNN은 수행시간이 많이 소요됨, 코드 테스트시에는 TRAINING EPOCHS를 줄여 실행할 필요가 있음.

물론 구글, 아마존 크라우드 서비스 활용도 가능함( 경우에 따라 비용발생)

그래픽 카드 (GPU NVIDIA )를 적용하는 방법도 있음.

"""


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 11 MNIST and Deep learning CNN

import tensorflow as tf

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# hyper parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100



class Model:


    def __init__(self, sess, name):

        self.sess = sess

        self.name = name

        self._build_net()


    def _build_net(self):

        with tf.variable_scope(self.name):

            # dropout (keep_prob) rate  0.7~0.5 on training, but should be 1

            # for testing

            self.training = tf.placeholder(tf.bool)


            # input place holders

            self.X = tf.placeholder(tf.float32, [None, 784])


            # img 28x28x1 (black/white), Input Layer

            X_img = tf.reshape(self.X, [-1, 28, 28, 1])

            self.Y = tf.placeholder(tf.float32, [None, 10])


            # Convolutional Layer #1

            conv1 = tf.layers.conv2d(inputs=X_img, filters=32, kernel_size=[3, 3],

                                     padding="SAME", activation=tf.nn.relu)

            # Pooling Layer #1

            pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2],

                                            padding="SAME", strides=2)

            dropout1 = tf.layers.dropout(inputs=pool1,

                                         rate=0.7, training=self.training)


            # Convolutional Layer #2 and Pooling Layer #2

            conv2 = tf.layers.conv2d(inputs=dropout1, filters=64, kernel_size=[3, 3],

                                     padding="SAME", activation=tf.nn.relu)

            pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2],

                                            padding="SAME", strides=2)

            dropout2 = tf.layers.dropout(inputs=pool2,

                                         rate=0.7, training=self.training)


            # Convolutional Layer #2 and Pooling Layer #2

            conv3 = tf.layers.conv2d(inputs=dropout2, filters=128, kernel_size=[3, 3],

                                     padding="same", activation=tf.nn.relu)

            pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2],

                                            padding="same", strides=2)

            dropout3 = tf.layers.dropout(inputs=pool3,

                                         rate=0.7, training=self.training)


            # Dense Layer with Relu

            flat = tf.reshape(dropout3, [-1, 128 * 4 * 4])

            dense4 = tf.layers.dense(inputs=flat,

                                     units=625, activation=tf.nn.relu)

            dropout4 = tf.layers.dropout(inputs=dense4,

                                         rate=0.5, training=self.training)


            # Logits (no activation) Layer: L5 Final FC 625 inputs -> 10 outputs

            self.logits = tf.layers.dense(inputs=dropout4, units=10)


        # define cost/loss & optimizer

        self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

            logits=self.logits, labels=self.Y))

        self.optimizer = tf.train.AdamOptimizer(

            learning_rate=learning_rate).minimize(self.cost)


        correct_prediction = tf.equal(

            tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))

        self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


    def predict(self, x_test, training=False):

        return self.sess.run(self.logits,

                             feed_dict={self.X: x_test, self.training: training})


    def get_accuracy(self, x_test, y_test, training=False):

        return self.sess.run(self.accuracy,

                             feed_dict={self.X: x_test,

                                        self.Y: y_test, self.training: training})


    def train(self, x_data, y_data, training=True):

        return self.sess.run([self.cost, self.optimizer], feed_dict={

            self.X: x_data, self.Y: y_data, self.training: training})


# initialize

sess = tf.Session()

m1 = Model(sess, "m1")


sess.run(tf.global_variables_initializer())


print('Learning Started!')


# train my model

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        c, _ = m1.train(batch_xs, batch_ys)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

print('Accuracy:', m1.get_accuracy(mnist.test.images, mnist.test.labels))




[ # lab-11-5-mnist_cnn_ensemble_layers ]


# lab-11-5-mnist_cnn_ensemble_layers


" 각각 예측시키고, 후에 병합을 해서 예측치를 출력시킴"


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 11 MNIST and Deep learning CNN

# https://www.tensorflow.org/tutorials/layers

import tensorflow as tf

import numpy as np


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# hyper parameters

learning_rate = 0.001

training_epochs = 20

batch_size = 100



class Model:


    def __init__(self, sess, name):

        self.sess = sess

        self.name = name

        self._build_net()


    def _build_net(self):

        with tf.variable_scope(self.name):

            # dropout (keep_prob) rate  0.7~0.5 on training, but should be 1

            # for testing

            self.training = tf.placeholder(tf.bool)


            # input place holders

            self.X = tf.placeholder(tf.float32, [None, 784])


            # img 28x28x1 (black/white), Input Layer

            X_img = tf.reshape(self.X, [-1, 28, 28, 1])

            self.Y = tf.placeholder(tf.float32, [None, 10])


            # Convolutional Layer #1

            conv1 = tf.layers.conv2d(inputs=X_img, filters=32, kernel_size=[3, 3],

                                     padding="SAME", activation=tf.nn.relu)

            # Pooling Layer #1

            pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2],

                                            padding="SAME", strides=2)

            dropout1 = tf.layers.dropout(inputs=pool1,

                                         rate=0.7, training=self.training)


            # Convolutional Layer #2 and Pooling Layer #2

            conv2 = tf.layers.conv2d(inputs=dropout1, filters=64, kernel_size=[3, 3],

                                     padding="SAME", activation=tf.nn.relu)

            pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2],

                                            padding="SAME", strides=2)

            dropout2 = tf.layers.dropout(inputs=pool2,

                                         rate=0.7, training=self.training)


            # Convolutional Layer #3 and Pooling Layer #3

            conv3 = tf.layers.conv2d(inputs=dropout2, filters=128, kernel_size=[3, 3],

                                     padding="SAME", activation=tf.nn.relu)

            pool3 = tf.layers.max_pooling2d(inputs=conv3, pool_size=[2, 2],

                                            padding="SAME", strides=2)

            dropout3 = tf.layers.dropout(inputs=pool3,

                                         rate=0.7, training=self.training)


            # Dense Layer with Relu

            flat = tf.reshape(dropout3, [-1, 128 * 4 * 4])

            dense4 = tf.layers.dense(inputs=flat,

                                     units=625, activation=tf.nn.relu)

            dropout4 = tf.layers.dropout(inputs=dense4,

                                         rate=0.5, training=self.training)


            # Logits (no activation) Layer: L5 Final FC 625 inputs -> 10 outputs

            self.logits = tf.layers.dense(inputs=dropout4, units=10)


        # define cost/loss & optimizer

        self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

            logits=self.logits, labels=self.Y))

        self.optimizer = tf.train.AdamOptimizer(

            learning_rate=learning_rate).minimize(self.cost)


        correct_prediction = tf.equal(

            tf.argmax(self.logits, 1), tf.argmax(self.Y, 1))

        self.accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))


    def predict(self, x_test, training=False):

        return self.sess.run(self.logits,

                             feed_dict={self.X: x_test, self.training: training})


    def get_accuracy(self, x_test, y_test, training=False):

        return self.sess.run(self.accuracy,

                             feed_dict={self.X: x_test,

                                        self.Y: y_test, self.training: training})


    def train(self, x_data, y_data, training=True):

        return self.sess.run([self.cost, self.optimizer], feed_dict={

            self.X: x_data, self.Y: y_data, self.training: training})


# initialize

sess = tf.Session()


models = []

num_models = 2

for m in range(num_models):

    models.append(Model(sess, "model" + str(m)))


sess.run(tf.global_variables_initializer())


print('Learning Started!')


# train my model

for epoch in range(training_epochs):

    avg_cost_list = np.zeros(len(models))

    total_batch = int(mnist.train.num_examples / batch_size)

    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)


        # train each model

        for m_idx, m in enumerate(models):

            c, _ = m.train(batch_xs, batch_ys)

            avg_cost_list[m_idx] += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', avg_cost_list)


print('Learning Finished!')


# Test model and check accuracy

test_size = len(mnist.test.labels)

predictions = np.zeros(test_size * 10).reshape(test_size, 10)

for m_idx, m in enumerate(models):

    print(m_idx, 'Accuracy:', m.get_accuracy(

        mnist.test.images, mnist.test.labels))

    p = m.predict(mnist.test.images)

    predictions += p


ensemble_correct_prediction = tf.equal(

    tf.argmax(predictions, 1), tf.argmax(mnist.test.labels, 1))

ensemble_accuracy = tf.reduce_mean(

    tf.cast(ensemble_correct_prediction, tf.float32))

print('Ensemble accuracy:', sess.run(ensemble_accuracy))


'''

0 Accuracy: 0.9933

1 Accuracy: 0.9946

2 Accuracy: 0.9934

3 Accuracy: 0.9935

4 Accuracy: 0.9935

5 Accuracy: 0.9949

6 Accuracy: 0.9941


Ensemble accuracy: 0.9952

'''




[ 참고자료 ]

  https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌

  https://github.com/hunkim/deeplearningzerotoall

  https://www.tensorflow.org/api_docs/python/tf/layers


Comment +0

인공지능 구현에 대한 글입니다.


글의 순서는 아래와 같습니다.


================================================

1. # lab-10-1-mnist_softmax 

    ==># 정확도 90%

2. # lab-10-2-mnist_nn

   3 단 깊이 /relu 적용=> 정확도 94 %

3. # lab-10-3-mnist_nn_xavier
   xavier 적용(초기값 세팅) ==> 정확도 97.6 %

4. #lab-10-4-mnist_nn_deep

   더 넓게(512), 더깊게 (L5) ==> 정확도 향상효과 없음

   원인: OVERFITTING  대책 : Drop out 적용

5. lab-10-5-mnist_nn_dropout
   drop out 적용 : 정확도 97.96%
6. 코드 탐구(추가)
   lab-10-6-mnist_nn_batchnorm
   lab-10-7-mnist_nn_higher_level_API
   lab-10-8-mnist_nn_selu(wip)
   lab-10-X1-mnist_back_prop

7. 참고자료

=================================================


[  lab-10-1-mnist_softmax]


# lab-10-1-mnist_softmax

# 정확도 90%

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 7 Learning rate and Evaluation

import tensorflow as tf

import random

#import matplotlib.pyplot as plt

from tensorflow.examples.tutorials.mnist import input_data

tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100


# input place holders

X = tf.placeholder(tf.float32, [None, 784])

Y = tf.placeholder(tf.float32, [None, 10])


# weights & bias for nn layers

W = tf.Variable(tf.random_normal([784, 10]))

b = tf.Variable(tf.random_normal([10]))


hypothesis = tf.matmul(X, W) + b


# define cost/loss & optimizer

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

    logits=hypothesis, labels=Y))

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)


# initialize

sess = tf.Session()

sess.run(tf.global_variables_initializer())


# train my model

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        feed_dict = {X: batch_xs, Y: batch_ys}

        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print('Accuracy:', sess.run(accuracy, feed_dict={

      X: mnist.test.images, Y: mnist.test.labels}))


# Get one and predict

r = random.randint(0, mnist.test.num_examples - 1)

print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

print("Prediction: ", sess.run(

    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))


#plt.imshow(mnist.test.images[r:r + 1].

 #   reshape(28, 28), cmap='Greys', interpolation='nearest')

#plt.show()


'''

Epoch: 0001 cost = 5.888845987

Epoch: 0002 cost = 1.860620173

Epoch: 0003 cost = 1.159035648

Epoch: 0004 cost = 0.892340870

Epoch: 0005 cost = 0.751155428

Epoch: 0006 cost = 0.662484806

Epoch: 0007 cost = 0.601544010

Epoch: 0008 cost = 0.556526115

Epoch: 0009 cost = 0.521186961

Epoch: 0010 cost = 0.493068354

Epoch: 0011 cost = 0.469686249

Epoch: 0012 cost = 0.449967254

Epoch: 0013 cost = 0.433519321

Epoch: 0014 cost = 0.419000337

Epoch: 0015 cost = 0.406490815

Learning Finished!

Accuracy: 0.9035

'''


[# lab-10-2-mnist_nn]

# lab-10-2-mnist_nn

# 3 단 깊이 /relu 적용=> 정확도 94 %

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 10 MNIST and NN

import tensorflow as tf

import random

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100


# input place holders

X = tf.placeholder(tf.float32, [None, 784])

Y = tf.placeholder(tf.float32, [None, 10])


# weights & bias for nn layers

W1 = tf.Variable(tf.random_normal([784, 256]))

b1 = tf.Variable(tf.random_normal([256]))

L1 = tf.nn.relu(tf.matmul(X, W1) + b1)


W2 = tf.Variable(tf.random_normal([256, 256]))

b2 = tf.Variable(tf.random_normal([256]))

L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)


W3 = tf.Variable(tf.random_normal([256, 10]))

b3 = tf.Variable(tf.random_normal([10]))

hypothesis = tf.matmul(L2, W3) + b3


# define cost/loss & optimizer

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

    logits=hypothesis, labels=Y))

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)


# initialize

sess = tf.Session()

sess.run(tf.global_variables_initializer())


# train my model

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        feed_dict = {X: batch_xs, Y: batch_ys}

        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print('Accuracy:', sess.run(accuracy, feed_dict={

      X: mnist.test.images, Y: mnist.test.labels}))


# Get one and predict

r = random.randint(0, mnist.test.num_examples - 1)

print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

print("Prediction: ", sess.run(

    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))


# plt.imshow(mnist.test.images[r:r + 1].

#           reshape(28, 28), cmap='Greys', interpolation='nearest')

# plt.show()


'''

Epoch: 0001 cost = 141.207671860

Epoch: 0002 cost = 38.788445864

Epoch: 0003 cost = 23.977515479

Epoch: 0004 cost = 16.315132428

Epoch: 0005 cost = 11.702554882

Epoch: 0006 cost = 8.573139748

Epoch: 0007 cost = 6.370995680

Epoch: 0008 cost = 4.537178684

Epoch: 0009 cost = 3.216900532

Epoch: 0010 cost = 2.329708954

Epoch: 0011 cost = 1.715552875

Epoch: 0012 cost = 1.189857912

Epoch: 0013 cost = 0.820965160

Epoch: 0014 cost = 0.624131458

Epoch: 0015 cost = 0.454633765

Learning Finished!

Accuracy: 0.9455


'''

[ lab-10-3-mnist_nn_xavier]

# lab-10-3-mnist_nn_xavier
# xavier 적용 ==> 정확도 97.6 %
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 10 MNIST and Xavier
import tensorflow as tf
import random
# import matplotlib.pyplot as plt

from tensorflow.examples.tutorials.mnist import input_data

tf.set_random_seed(777)  # reproducibility

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset

# parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100

# input place holders
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])

# weights & bias for nn layers
# http://stackoverflow.com/questions/33640581/how-to-do-xavier-initialization-on-tensorflow
W1 = tf.get_variable("W1", shape=[784, 256],
                     initializer=tf.contrib.layers.xavier_initializer())
b1 = tf.Variable(tf.random_normal([256]))
L1 = tf.nn.relu(tf.matmul(X, W1) + b1)

W2 = tf.get_variable("W2", shape=[256, 256],
                     initializer=tf.contrib.layers.xavier_initializer())
b2 = tf.Variable(tf.random_normal([256]))
L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)

W3 = tf.get_variable("W3", shape=[256, 10],
                     initializer=tf.contrib.layers.xavier_initializer())
b3 = tf.Variable(tf.random_normal([10]))
hypothesis = tf.matmul(L2, W3) + b3

# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
    logits=hypothesis, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())

# train my model
for epoch in range(training_epochs):
    avg_cost = 0
    total_batch = int(mnist.train.num_examples / batch_size)

    for i in range(total_batch):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size)
        feed_dict = {X: batch_xs, Y: batch_ys}
        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
        avg_cost += c / total_batch

    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))

print('Learning Finished!')

# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
      X: mnist.test.images, Y: mnist.test.labels}))

# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))

# plt.imshow(mnist.test.images[r:r + 1].
#           reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()

'''
Epoch: 0001 cost = 0.301498963
Epoch: 0002 cost = 0.107252513
Epoch: 0003 cost = 0.064888892
Epoch: 0004 cost = 0.044463030
Epoch: 0005 cost = 0.029951642
Epoch: 0006 cost = 0.020663404
Epoch: 0007 cost = 0.015853033
Epoch: 0008 cost = 0.011764387
Epoch: 0009 cost = 0.008598264
Epoch: 0010 cost = 0.007383116
Epoch: 0011 cost = 0.006839140
Epoch: 0012 cost = 0.004672963
Epoch: 0013 cost = 0.003979437
Epoch: 0014 cost = 0.002714260
Epoch: 0015 cost = 0.004707661
Learning Finished!
Accuracy: 0.9783
'''
[ lab-10-4-mnist_nn_deep ]

#lab-10-4-mnist_nn_deep

# 더 넓게(512), 더깊게 (L5) ==> 정확도 향상효과 없음

# 원인: OVERFITTING  대책 : Drop out 적용



import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 10 MNIST and Deep learning

import tensorflow as tf

import random

# import matplotlib.pyplot as plt


from tensorflow.examples.tutorials.mnist import input_data


tf.set_random_seed(777)  # reproducibility


mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset


# parameters

learning_rate = 0.001

training_epochs = 15

batch_size = 100


# input place holders

X = tf.placeholder(tf.float32, [None, 784])

Y = tf.placeholder(tf.float32, [None, 10])


# weights & bias for nn layers

# http://stackoverflow.com/questions/33640581/how-to-do-xavier-initialization-on-tensorflow

W1 = tf.get_variable("W1", shape=[784, 512],

                     initializer=tf.contrib.layers.xavier_initializer())

b1 = tf.Variable(tf.random_normal([512]))

L1 = tf.nn.relu(tf.matmul(X, W1) + b1)


W2 = tf.get_variable("W2", shape=[512, 512],

                     initializer=tf.contrib.layers.xavier_initializer())

b2 = tf.Variable(tf.random_normal([512]))

L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)


W3 = tf.get_variable("W3", shape=[512, 512],

                     initializer=tf.contrib.layers.xavier_initializer())

b3 = tf.Variable(tf.random_normal([512]))

L3 = tf.nn.relu(tf.matmul(L2, W3) + b3)


W4 = tf.get_variable("W4", shape=[512, 512],

                     initializer=tf.contrib.layers.xavier_initializer())

b4 = tf.Variable(tf.random_normal([512]))

L4 = tf.nn.relu(tf.matmul(L3, W4) + b4)


W5 = tf.get_variable("W5", shape=[512, 10],

                     initializer=tf.contrib.layers.xavier_initializer())

b5 = tf.Variable(tf.random_normal([10]))

hypothesis = tf.matmul(L4, W5) + b5


# define cost/loss & optimizer

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(

    logits=hypothesis, labels=Y))

optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)


# initialize

sess = tf.Session()

sess.run(tf.global_variables_initializer())


# train my model

for epoch in range(training_epochs):

    avg_cost = 0

    total_batch = int(mnist.train.num_examples / batch_size)


    for i in range(total_batch):

        batch_xs, batch_ys = mnist.train.next_batch(batch_size)

        feed_dict = {X: batch_xs, Y: batch_ys}

        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)

        avg_cost += c / total_batch


    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))


print('Learning Finished!')


# Test model and check accuracy

correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

print('Accuracy:', sess.run(accuracy, feed_dict={

      X: mnist.test.images, Y: mnist.test.labels}))


# Get one and predict

r = random.randint(0, mnist.test.num_examples - 1)

print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

print("Prediction: ", sess.run(

    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))


# plt.imshow(mnist.test.images[r:r + 1].

#           reshape(28, 28), cmap='Greys', interpolation='nearest')

# plt.show()


'''

Epoch: 0001 cost = 0.266061549

Epoch: 0002 cost = 0.080796588

Epoch: 0003 cost = 0.049075800

Epoch: 0004 cost = 0.034772298

Epoch: 0005 cost = 0.024780529

Epoch: 0006 cost = 0.017072763

Epoch: 0007 cost = 0.014031383

Epoch: 0008 cost = 0.013763446

Epoch: 0009 cost = 0.009164047

Epoch: 0010 cost = 0.008291388

Epoch: 0011 cost = 0.007319742

Epoch: 0012 cost = 0.006434021

Epoch: 0013 cost = 0.005684378

Epoch: 0014 cost = 0.004781207

Epoch: 0015 cost = 0.004342310

Learning Finished!

Accuracy: 0.9742

'''


[ lab-10-5-mnist_nn_dropout ]

#lab-10-5-mnist_nn_dropout
# drop out 적용 : 정확도 97.96%
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
# Lab 10 MNIST and Dropout
import tensorflow as tf
import random
# import matplotlib.pyplot as plt

from tensorflow.examples.tutorials.mnist import input_data

tf.set_random_seed(777)  # reproducibility

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
# Check out https://www.tensorflow.org/get_started/mnist/beginners for
# more information about the mnist dataset

# parameters
learning_rate = 0.001
training_epochs = 15
batch_size = 100

# input place holders
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])

# dropout (keep_prob) rate  0.7 on training, but should be 1 for testing
keep_prob = tf.placeholder(tf.float32)

# weights & bias for nn layers
# http://stackoverflow.com/questions/33640581/how-to-do-xavier-initialization-on-tensorflow
W1 = tf.get_variable("W1", shape=[784, 512],
                     initializer=tf.contrib.layers.xavier_initializer())
b1 = tf.Variable(tf.random_normal([512]))
L1 = tf.nn.relu(tf.matmul(X, W1) + b1)
L1 = tf.nn.dropout(L1, keep_prob=keep_prob)

W2 = tf.get_variable("W2", shape=[512, 512],
                     initializer=tf.contrib.layers.xavier_initializer())
b2 = tf.Variable(tf.random_normal([512]))
L2 = tf.nn.relu(tf.matmul(L1, W2) + b2)
L2 = tf.nn.dropout(L2, keep_prob=keep_prob)

W3 = tf.get_variable("W3", shape=[512, 512],
                     initializer=tf.contrib.layers.xavier_initializer())
b3 = tf.Variable(tf.random_normal([512]))
L3 = tf.nn.relu(tf.matmul(L2, W3) + b3)
L3 = tf.nn.dropout(L3, keep_prob=keep_prob)

W4 = tf.get_variable("W4", shape=[512, 512],
                     initializer=tf.contrib.layers.xavier_initializer())
b4 = tf.Variable(tf.random_normal([512]))
L4 = tf.nn.relu(tf.matmul(L3, W4) + b4)
L4 = tf.nn.dropout(L4, keep_prob=keep_prob)

W5 = tf.get_variable("W5", shape=[512, 10],
                     initializer=tf.contrib.layers.xavier_initializer())
b5 = tf.Variable(tf.random_normal([10]))
hypothesis = tf.matmul(L4, W5) + b5

# define cost/loss & optimizer
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
    logits=hypothesis, labels=Y))
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

# initialize
sess = tf.Session()
sess.run(tf.global_variables_initializer())

# train my model
for epoch in range(training_epochs):
    avg_cost = 0
    total_batch = int(mnist.train.num_examples / batch_size)

    for i in range(total_batch):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size)
        feed_dict = {X: batch_xs, Y: batch_ys, keep_prob: 0.7}
        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)
        avg_cost += c / total_batch

    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))

print('Learning Finished!')

# Test model and check accuracy
correct_prediction = tf.equal(tf.argmax(hypothesis, 1), tf.argmax(Y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Accuracy:', sess.run(accuracy, feed_dict={
      X: mnist.test.images, Y: mnist.test.labels, keep_prob: 1}))

# Get one and predict
r = random.randint(0, mnist.test.num_examples - 1)
print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))
print("Prediction: ", sess.run(
    tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1], keep_prob: 1}))

# plt.imshow(mnist.test.images[r:r + 1].
#           reshape(28, 28), cmap='Greys', interpolation='nearest')
# plt.show()

'''
Epoch: 0001 cost = 0.447322626
Epoch: 0002 cost = 0.157285590
Epoch: 0003 cost = 0.121884535
Epoch: 0004 cost = 0.098128681
Epoch: 0005 cost = 0.082901778
Epoch: 0006 cost = 0.075337573
Epoch: 0007 cost = 0.069752543
Epoch: 0008 cost = 0.060884363
Epoch: 0009 cost = 0.055276413
Epoch: 0010 cost = 0.054631256
Epoch: 0011 cost = 0.049675195
Epoch: 0012 cost = 0.049125314
Epoch: 0013 cost = 0.047231930
Epoch: 0014 cost = 0.041290121
Epoch: 0015 cost = 0.043621063
Learning Finished!
Accuracy: 0.9804
'''

[ 참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌

https://github.com/hunkim/deeplearningzerotoall



Comment +0

인공지능 구현에 대한 글입니다.


글의 순서는 아래와 같습니다.


================================================

1. # lab-09-1-xor 

    ==># 코딩은 이상이 없으나, 정확도는 50% 수준밖에는 않됩니다.

2. # lab-09-2-xor-nn

   ==>정확도 50%를 개선하는 방법으로 neural network ( 신경망 )을 이용하는 방법을 구현 ==> 정확도 100%

  ==>layer 1 이 다른곳에 입력으로 들어감 : 신경망 구성

    layer1은 출력 2개 로 구성한다 , 이유는 최종 가설의 입력(2개의 data)으로 들어가기 때문임

    정확도 1.0으로 개선이 됨

3. # lab-09-3-xor-nn-wide-deep

  ==> layer1의 출력을 10개  넓게 펴주고, 최종 가설에는 출력을 1개로 설정함 

       layer1,2,3로 더 깊게 만들고, 최종 출력을 1개로 설정함

  ==> 즉 더 넓게, 깊게 신경망을 구성해서  정확도 1.0으로 개선됨, 정밀도 높아짐 (큰것은 더 크게, 작은것은 더 작게)

4. 코딩탐구 필요(추가)

  ==> lab-09-4-xor_tensorboard

       lab-09-5-linear_back_prop

       lab-09-6-multi-linear_back_prop

       lab-09-7-sigmoid_back_prop

       lab-09-x-xor-nn-back_prop

5. 참고자료

=================================================


[rate overfitting , regularization tips]


1. COST 값이 줄지않고, 늘어난다면  Learning rate을 더 작게 변경해주어야함

   반대로 너무 작게 줄거나, 도중에 멈춘다면 learing rate을 좀더 크게 해주어야함

   보통 0.01을 기준으로 늘리거나 줄이면서 조정해 나가면 된다.


2.  x data 값이 차이가 큰 경우, cost가 잘 줄지않거나, 학습이 잘 일어나지 않음 , 이럴때는 NOMALIZATION 을 해주어야 함.

3. OVERFITTING

  - 더많은 TRAINING DATA

  - REDUCE FEATURES

  - REGULARIZATION ==> 구별선이 구부러지지 않토록 , X값을 일반화 시킴


[lab-09-1-xor]

# lab-09-1-xor

# 코딩은 이상이 없으나, 정확도는 50% 수준밖에는 않됩니다.



import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 9 XOR

import tensorflow as tf

import numpy as np


tf.set_random_seed(777)  # for reproducibility

learning_rate = 0.1


x_data = [[0, 0],

          [0, 1],

          [1, 0],

          [1, 1]]

y_data = [[0],

          [1],

          [1],

          [0]]

x_data = np.array(x_data, dtype=np.float32)

y_data = np.array(y_data, dtype=np.float32)


X = tf.placeholder(tf.float32, [None, 2])

Y = tf.placeholder(tf.float32, [None, 1])


W = tf.Variable(tf.random_normal([2, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))

hypothesis = tf.sigmoid(tf.matmul(X, W) + b)


# cost/loss function

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *

                       tf.log(1 - hypothesis))


train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)


# Accuracy computation

# True if hypothesis>0.5 else False

predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)

accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(10001):

        sess.run(train, feed_dict={X: x_data, Y: y_data})

        if step % 100 == 0:

            print(step, sess.run(cost, feed_dict={

                  X: x_data, Y: y_data}), sess.run(W))


    # Accuracy report

    h, c, a = sess.run([hypothesis, predicted, accuracy],

                       feed_dict={X: x_data, Y: y_data})

    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)


'''

Hypothesis:  [[ 0.5]

 [ 0.5]

 [ 0.5]

 [ 0.5]]

Correct:  [[ 0.]

 [ 0.]

 [ 0.]

 [ 0.]]

Accuracy:  0.5


'''

[ lab-09-2-xor-nn ]

# lab-09-2-xor-nn

""" 정확도 50%를 개선하는 방법으로 neural network ( 신경망 )을 이용하는 방법을 구현하함"""

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 9 XOR

import tensorflow as tf

import numpy as np


tf.set_random_seed(777)  # for reproducibility

learning_rate = 0.1


x_data = [[0, 0],

          [0, 1],

          [1, 0],

          [1, 1]]

y_data = [[0],

          [1],

          [1],

          [0]]

x_data = np.array(x_data, dtype=np.float32)

y_data = np.array(y_data, dtype=np.float32)


X = tf.placeholder(tf.float32, [None, 2])

Y = tf.placeholder(tf.float32, [None, 1])


W1 = tf.Variable(tf.random_normal([2, 2]), name='weight1')

b1 = tf.Variable(tf.random_normal([2]), name='bias1')

layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)


""" layer 1 이 다른곳에 입력으로 연결함 : 신경망 구성

    layer1은 입력1개, 출력 2로 구성한다 , 이유는 최종 가설의 입력으로 들어가기 때문임

    정확도 1.0으로 개선이 됨

"""

W2 = tf.Variable(tf.random_normal([2, 1]), name='weight2')

b2 = tf.Variable(tf.random_normal([1]), name='bias2')

hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)


# cost/loss function

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *

                       tf.log(1 - hypothesis))


train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)


# Accuracy computation

# True if hypothesis>0.5 else False

predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)

accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(10001):

        sess.run(train, feed_dict={X: x_data, Y: y_data})

        if step % 100 == 0:

            print(step, sess.run(cost, feed_dict={

                  X: x_data, Y: y_data}), sess.run([W1, W2]))


    # Accuracy report

    h, c, a = sess.run([hypothesis, predicted, accuracy],

                       feed_dict={X: x_data, Y: y_data})

    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)



'''

Hypothesis:  [[ 0.01338218]

 [ 0.98166394]

 [ 0.98809403]

 [ 0.01135799]]

Correct:  [[ 0.]

 [ 1.]

 [ 1.]

 [ 0.]]

Accuracy:  1.0

'''


[ lab-09-3-xor-nn-wide-deep ]


# lab-09-3-xor-nn-wide-deep


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 9 XOR

import tensorflow as tf

import numpy as np


tf.set_random_seed(777)  # for reproducibility

learning_rate = 0.1


x_data = [[0, 0],

          [0, 1],

          [1, 0],

          [1, 1]]

y_data = [[0],

          [1],

          [1],

          [0]]

x_data = np.array(x_data, dtype=np.float32)

y_data = np.array(y_data, dtype=np.float32)


X = tf.placeholder(tf.float32, [None, 2])

Y = tf.placeholder(tf.float32, [None, 1])


W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1')

b1 = tf.Variable(tf.random_normal([10]), name='bias1')

layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)

""" layer1의 출력을 10개  넓게 펴주고, 최종 가설에는 출력을 1개로 설정함 

    layer1,2,3로 더 깊게 만들고, 최종 출력을 1개로 설정함

    ==> 즉 더 넓게, 깊게 신경망을 구성해서  정확도 1.0으로 개선됨 """


W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')

b2 = tf.Variable(tf.random_normal([10]), name='bias2')

layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)


W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')

b3 = tf.Variable(tf.random_normal([10]), name='bias3')

layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)


W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')

b4 = tf.Variable(tf.random_normal([1]), name='bias4')

hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)


# cost/loss function

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *

                       tf.log(1 - hypothesis))


train = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cost)


# Accuracy computation

# True if hypothesis>0.5 else False

predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)

accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(10001):

        sess.run(train, feed_dict={X: x_data, Y: y_data})

        if step % 100 == 0:

            print(step, sess.run(cost, feed_dict={

                  X: x_data, Y: y_data}), sess.run([W1, W2]))


    # Accuracy report

    h, c, a = sess.run([hypothesis, predicted, accuracy],

                       feed_dict={X: x_data, Y: y_data})

    print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)



'''

Hypothesis:  [[  7.80511764e-04]

 [  9.99238133e-01]

 [  9.98379230e-01]

 [  1.55659032e-03]]

Correct:  [[ 0.]

 [ 1.]

 [ 1.]

 [ 0.]]

Accuracy:  1.0

'''


[참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌

https://github.com/hunkim/deeplearningzerotoall




Comment +0

인공지능 구현에 대한 글입니다.

글의 순서는 아래와 같습니다.


================================================

1. rate overfitting , regularization tips

2. train/test data sheet , learning rate , normalization(new)

 - 학습과 테스트 data를 구분하는것이 합리적임

 - mnist 소개

3. 참고자료

=================================================


[rate overfitting , regularization tips]


1. COST 값이 줄지않고, 늘어난다면  Learning rate을 더 작게 변경해주어야함

   반대로 너무 작게 줄거나, 도중에 멈춘다면 learing rate을 좀더 크게 해주어야함

   보통 0.01을 기준으로 늘리거나 줄이면서 조정해 나가면 된다.


2.  x data 값이 차이가 큰 경우, cost가 잘 줄지않거나, 학습이 잘 일어나지 않음 , 이럴때는 NOMALIZATION 을 해주어야 함.

3. OVERFITTING

  - 더많은 TRAINING DATA

  - REDUCE FEATURES

  - REGULARIZATION ==> 구별선이 구부러지지 않토록 , X값을 일반화 시킴


[train/test data sheet , learning rate , normalization(new)]


# <lab-07-1-learning_rate_and_evaluation>


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 7 Learning rate and Evaluation

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


x_data = [[1, 2, 1],

          [1, 3, 2],

          [1, 3, 4],

          [1, 5, 5],

          [1, 7, 5],

          [1, 2, 5],

          [1, 6, 6],

          [1, 7, 7]]

y_data = [[0, 0, 1],

          [0, 0, 1],

          [0, 0, 1],

          [0, 1, 0],

          [0, 1, 0],

          [0, 1, 0],

          [1, 0, 0],

          [1, 0, 0]]


# Evaluation our model using this test dataset


x_test = [[2, 1, 1],

          [3, 1, 2],

          [3, 3, 4]]

y_test = [[0, 0, 1],

          [0, 0, 1],

          [0, 0, 1]]


X = tf.placeholder("float", [None, 3])

Y = tf.placeholder("float", [None, 3])


W = tf.Variable(tf.random_normal([3, 3]))

b = tf.Variable(tf.random_normal([3]))


# tf.nn.softmax computes softmax activations

# softmax = exp(logits) / reduce_sum(exp(logits), dim)

hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)


# Cross entropy cost/loss

cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))

# Try to change learning_rate to small numbers

optimizer = tf.train.GradientDescentOptimizer(

    learning_rate=0.1).minimize(cost)


# dhp Learning rate을 1.5로 움직이는 크기를 크게하면, H(y)값은 발산을 하게된다 : overfitting )


# Correct prediction Test model

prediction = tf.arg_max(hypothesis, 1)

is_correct = tf.equal(prediction, tf.arg_max(Y, 1))

accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(201):

        cost_val, W_val, _ = sess.run(

            [cost, W, optimizer], feed_dict={X: x_data, Y: y_data})

        print(step, cost_val, W_val)


# 여기부터는 test data 로 test 진행함

# tf 입장에서는 처움보는 data (x)가 되는것이다.


    # predict

    print("Prediction:", sess.run(prediction, feed_dict={X: x_test}))

    # Calculate the accuracy

    print("Accuracy: ", sess.run(accuracy, feed_dict={X: x_test, Y: y_test}))


'''

when lr = 1.5

0 5.73203 [[-0.30548954  1.22985029 -0.66033536]

 [-4.39069986  2.29670858  2.99386835]

 [-3.34510708  2.09743214 -0.80419564]]

1 23.1494 [[ 0.06951046  0.29449689 -0.0999819 ]

 [-1.95319986 -1.63627958  4.48935604]

 [-0.90760708 -1.65020132  0.50593793]]

2 27.2798 [[ 0.44451016  0.85699677 -1.03748143]

 [ 0.48429942  0.98872018 -0.57314301]

 [ 1.52989244  1.16229868 -4.74406147]]

3 8.668 [[ 0.12396193  0.61504567 -0.47498202]

 [ 0.22003263 -0.2470119   0.9268558 ]

 [ 0.96035379  0.41933775 -3.43156195]]

4 5.77111 [[-0.9524312   1.13037777  0.08607888]

 [-3.78651619  2.26245379  2.42393875]

 [-3.07170963  3.14037919 -2.12054014]]

5 inf [[ nan  nan  nan]

 [ nan  nan  nan]

 [ nan  nan  nan]]

6 nan [[ nan  nan  nan]

 [ nan  nan  nan]

 [ nan  nan  nan]]


 ...

Prediction: [0 0 0]

Accuracy:  0.0


-------------------------------------------------

When lr = 1e-10

0 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

1 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

2 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

...


198 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

199 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

200 5.73203 [[ 0.80269563  0.67861295 -1.21728313]

 [-0.3051686  -0.3032113   1.50825703]

 [ 0.75722361 -0.7008909  -2.10820389]]

Prediction: [0 0 0]

Accuracy:  0.0

-------------------------------------------------

When lr = 0.1


0 5.73203 [[ 0.72881663  0.71536207 -1.18015325]

 [-0.57753736 -0.12988332  1.60729778]

 [ 0.48373488 -0.51433605 -2.02127004]]

1 3.318 [[ 0.66219079  0.74796319 -1.14612854]

 [-0.81948912  0.03000021  1.68936598]

 [ 0.23214608 -0.33772916 -1.94628811]]

2 2.0218 [[ 0.64342022  0.74127686 -1.12067163]

 [-0.81161296 -0.00900121  1.72049117]

 [ 0.2086665  -0.35079569 -1.909742  ]]


...


199 0.672261 [[-1.15377033  0.28146935  1.13632679]

 [ 0.37484586  0.18958236  0.33544877]

 [-0.35609841 -0.43973011 -1.25604188]]

200 0.670909 [[-1.15885413  0.28058422  1.14229572]

 [ 0.37609792  0.19073224  0.33304682]

 [-0.35536593 -0.44033223 -1.2561723 ]]

Prediction: [2 2 2]

Accuracy:  1.0

'''


# < lab-07-2-linear_regression_without_min_max >


# dhp data 편차가 터무니없이 크게되면, 학습이 잘 되질 않됨


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

import tensorflow as tf

import numpy as np

tf.set_random_seed(777)  # for reproducibility



xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],

               [823.02002, 828.070007, 1828100, 821.655029, 828.070007],

               [819.929993, 824.400024, 1438100, 818.97998, 824.159973],

               [816, 820.958984, 1008100, 815.48999, 819.23999],

               [819.359985, 823, 1188100, 818.469971, 818.97998],

               [819, 823, 1198100, 816, 820.450012],

               [811.700012, 815.25, 1098100, 809.780029, 813.669983],

               [809.51001, 816.659973, 1398100, 804.539978, 809.559998]])


x_data = xy[:, 0:-1]

y_data = xy[:, [-1]]


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 4])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([4, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis

hypothesis = tf.matmul(X, W) + b


# Simplified cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(101):

    cost_val, hy_val, _ = sess.run(

        [cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})

    print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)



'''

0 Cost:  2.45533e+12

Prediction:

 [[-1104436.375]

 [-2224342.75 ]

 [-1749606.75 ]

 [-1226179.375]

 [-1445287.125]

 [-1457459.5  ]

 [-1335740.5  ]

 [-1700924.625]]

1 Cost:  2.69762e+27

Prediction:

 [[  3.66371490e+13]

 [  7.37543360e+13]

 [  5.80198785e+13]

 [  4.06716290e+13]

 [  4.79336847e+13]

 [  4.83371348e+13]

 [  4.43026590e+13]

 [  5.64060907e+13]]

2 Cost:  inf

Prediction:

 [[ -1.21438790e+21]

 [ -2.44468702e+21]

 [ -1.92314724e+21]

 [ -1.34811610e+21]

 [ -1.58882674e+21]

 [ -1.60219962e+21]

 [ -1.46847142e+21]

 [ -1.86965602e+21]]

3 Cost:  inf

Prediction:

 [[  4.02525216e+28]

 [  8.10324465e+28]

 [  6.37453079e+28]

 [  4.46851237e+28]

 [  5.26638074e+28]

 [  5.31070676e+28]

 [  4.86744608e+28]

 [  6.19722623e+28]]

4 Cost:  inf

Prediction:

 [[ -1.33422428e+36]

 [ -2.68593010e+36]

 [ -2.11292430e+36]

 [ -1.48114879e+36]

 [ -1.74561303e+36]

 [ -1.76030542e+36]

 [ -1.61338091e+36]

 [ -2.05415459e+36]]

5 Cost:  inf

Prediction:

 [[ inf]

 [ inf]

 [ inf]

 [ inf]

 [ inf]

 [ inf]

 [ inf]

 [ inf]]

6 Cost:  nan

Prediction:

 [[ nan]

 [ nan]

 [ nan]

 [ nan]

 [ nan]

 [ nan]

 [ nan]

 [ nan]]

'''



# <lab-07-3-linear_regression_min_max >


# dhp minmaxscala 함수를 이용해서, 정규화 (0~1사이의수)로 변환을 시킨다(nomallize)

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

import tensorflow as tf

import numpy as np

tf.set_random_seed(777)  # for reproducibility



def MinMaxScaler(data):

    numerator = data - np.min(data, 0)

    denominator = np.max(data, 0) - np.min(data, 0)

    # noise term prevents the zero division

    return numerator / (denominator + 1e-7)



xy = np.array([[828.659973, 833.450012, 908100, 828.349976, 831.659973],

               [823.02002, 828.070007, 1828100, 821.655029, 828.070007],

               [819.929993, 824.400024, 1438100, 818.97998, 824.159973],

               [816, 820.958984, 1008100, 815.48999, 819.23999],

               [819.359985, 823, 1188100, 818.469971, 818.97998],

               [819, 823, 1198100, 816, 820.450012],

               [811.700012, 815.25, 1098100, 809.780029, 813.669983],

               [809.51001, 816.659973, 1398100, 804.539978, 809.559998]])


# very important. It does not work without it.

xy = MinMaxScaler(xy)

print(xy)


x_data = xy[:, 0:-1]

y_data = xy[:, [-1]]


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 4])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([4, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis

hypothesis = tf.matmul(X, W) + b


# Simplified cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(101):

    cost_val, hy_val, _ = sess.run(

        [cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})

    print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)


'''

100 Cost:  0.152254

Prediction:

 [[ 1.63450289]

 [ 0.06628087]

 [ 0.35014752]

 [ 0.67070574]



# <lab-07-4-mnist_introduction>


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 7 Learning rate and Evaluation

import tensorflow as tf

import random

# import matplotlib.pyplot as plt

tf.set_random_seed(777)  # for reproducibility


from tensorflow.examples.tutorials.mnist import input_data

# Check out https://www.tensorflow.org/get_started/mnist/beginners for

# more information about the mnist dataset

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)


nb_classes = 10


# MNIST data image of shape 28 * 28 = 784

X = tf.placeholder(tf.float32, [None, 784])

# 0 - 9 digits recognition = 10 classes

Y = tf.placeholder(tf.float32, [None, nb_classes])


W = tf.Variable(tf.random_normal([784, nb_classes]))

b = tf.Variable(tf.random_normal([nb_classes]))


# Hypothesis (using softmax)

hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)


cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)


# Test model

is_correct = tf.equal(tf.arg_max(hypothesis, 1), tf.arg_max(Y, 1))

# Calculate accuracy

accuracy = tf.reduce_mean(tf.cast(is_correct, tf.float32))


# parameters

training_epochs = 15

batch_size = 100


with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())

    # Training cycle

    for epoch in range(training_epochs):

        avg_cost = 0

        total_batch = int(mnist.train.num_examples / batch_size)


        for i in range(total_batch):

            batch_xs, batch_ys = mnist.train.next_batch(batch_size)

            c, _ = sess.run([cost, optimizer], feed_dict={

                            X: batch_xs, Y: batch_ys})

            avg_cost += c / total_batch


        print('Epoch:', '%04d' % (epoch + 1),

              'cost =', '{:.9f}'.format(avg_cost))


    print("Learning finished")


    # Test the model using test sets

    print("Accuracy: ", accuracy.eval(session=sess, feed_dict={

          X: mnist.test.images, Y: mnist.test.labels}))


    # Get one and predict

    r = random.randint(0, mnist.test.num_examples - 1)

    print("Label: ", sess.run(tf.argmax(mnist.test.labels[r:r + 1], 1)))

    print("Prediction: ", sess.run(

        tf.argmax(hypothesis, 1), feed_dict={X: mnist.test.images[r:r + 1]}))


    # 그래프 인쇄가 않됨, 향후 원인 파악필요함, 일단은 #로 주석처리 함 

    # don't know why this makes Travis Build error.

    # plt.imshow(

    #     mnist.test.images[r:r + 1].reshape(28, 28),

    #     cmap='Greys',

    #     interpolation='nearest')

    # plt.show()



'''

Epoch: 0001 cost = 2.868104637

Epoch: 0002 cost = 1.134684615

Epoch: 0003 cost = 0.908220728

Epoch: 0004 cost = 0.794199896

Epoch: 0005 cost = 0.721815854

Epoch: 0006 cost = 0.670184430

Epoch: 0007 cost = 0.630576546

Epoch: 0008 cost = 0.598888191

Epoch: 0009 cost = 0.573027079

Epoch: 0010 cost = 0.550497213



[참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌

https://github.com/hunkim/deeplearningzerotoall

Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid

python 문법에 관한 글 입니다.

인공지능 tensor flow 코딩을 위해 python 문법에 대해 정리를 하고자 합니다.

글의 순서는 아래와 같습니다

==========================================

1. cheatsheet

 - python문법의 컨닝 페이퍼입니다.

2. 주제별 코딩예제 모음

3.  참고한 자료

==========================================

[cheatsheet]

cheatsheet.pdf

[주제별 코딩예제 모음]

auth.py

egoing.py

k8805.py

lib.py

python grammar.py

print('---수와 계산------------------------------------------------------------------------')
print(10 + 5)
print(10 - 5)
print(10 * 5)
print(10 / 5)

import math
print(math.ceil(2.2))
print(math.floor(2.7))
print(math.pow(2,10))
print(math.pi)

print('---문자와 데이터 타입----------------------------------------------------------------')
print('Hello')
print("Hello")
print("Hello 'world'")
print('Hello "world"')
print('Hello '+'world')
print('Hello '*3)
print('Hello'[0])
print('Hello'[1])
print('Hello'[2])

print('hello world'.capitalize())
print('hello world'.upper())
print('hello world'.__len__())
print(len('hello world'))
print('Hello world'.replace('world', 'programming'))

print("egoing's \"tutorial\"")
print("\\")
print("Hello\nworld")
print("Hello\t\tworld")
print("\a")
print('Hello\nworld')

print(10+5)
print("10"+"5")

print('---변수---------------------------------------------------------------------------')
x = 10
y = 5
print(x + y)

title = "python & ruby"
print("Title is " + title)

print('---문자열에서 변수의 사용---------------------------------------------------------------------------')
name = "이상효"
print("안녕하세요. "+name+"님")
print(name+"님을 위한 강의를 준비했습니다.")
print(name+"님 꼭 참석 부탁드립니다.")


print('---수계산 에서 변수의 사용---------------------------------------------------------------------------')
donation = 200
student = 10
sponsor = 100
print((donation*student)/sponsor)

print('---비교와 불리언---------------------------------------------------------------------------')
a=1
b=1
print(a==b)
print(1==2)
print(1>2)
print(1<2)
print(True)
print(False)

print('---조건문---------------------------------------------------------------------------')
if True:
print("code1")
print("code2")
print("code3")

print('---조건문의 활용---------------------------------------------------------------------------')
input = 11
real = 11
if real == input:
print("Hello!")

input = 11
real = 21
if real == input:
print("Hello!")
else:
print("Who are you?")

input = 33
real_egoing = 11
real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
elif real_k8805 == input:
print("Hello!, k8805")
else:
print("Who are you?")

print('---입력과 출력---------------------------------------------------------------------------')
in_str = input("입력 해주세요.\n")
print(in_str.upper()+" World!")

print('---로그인 애플리케이션에 입력기능 추가하기-------------------------------------------------------------------------')
in_str = input("1 아이디를 입력해주세요.\n")
real_egoing = "11"
real_k8805 = "ab"
if real_egoing == in_str:
print("Hello!, egoing")
elif real_k8805 == in_str:
print("Hello!, k8805")
else:
print("Who are you?")

print('---논리 연산---------------------------------------------------------------------------')
in_str = input("2 아이디를 입력해주세요.\n")
real_egoing = "egoing"
real_k8805 = "k8805"
if real_egoing == in_str or real_k8805 == in_str:
print("Hello!")
else:
print("Who are you?")

input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id:
if real_pwd == input_pwd:
print("Hello!")
else:
print("잘못된 비밀번호입니다")
else:
print("잘못된 아이디입니다")

input_id = input("아이디를 입력해주세요.\n")
input_pwd = input("비밀번호를 입력해주세요.\n")
real_id = "egoing"
real_pwd = "11"
if real_id == input_id and real_pwd == input_pwd:
print("Hello!")
else:
print("로그인에 실패했습니다")

print('---주석---------------------------------------------------------------------------')
'''
조건문 예제
egoing
2015
'''
# user input password
input = 33
real_egoing = 11
#real_k8805 = "ab"
if real_egoing == input:
print("Hello!, egoing")
#elif real_k8805 == input:
# print("Hello!, k8805")
else:
print("Who are you?")

print('---컨테이너---------------------------------------------------------------------------')
print(type('egoing')) # <class 'str'>
name = 'egoing'
print(name) # egoing
print(type(['egoing', 'leezche', 'graphittie'])) # <class 'list'>
names = ['egoing', 'leezche', 'graphittie']
print(names)
print(names[2]) # graphittie
egoing = ['programmer', 'seoul', 25, False]
egoing[1] = 'busan'
print(egoing) # ['programmer', 'busan', 25, False]

print('---사용설명서---------------------------------------------------------------------------')
al = ['A', 'B', 'C', 'D']
print(len(al)) # 4
al.append('E')
print(al) # ['A', 'B', 'C', 'D', 'E']
del (al[0])
print(al) # ['B', 'C', 'D', 'E']

print('---반복문---------------------------------------------------------------------------')
while False:
print('Hello world')
print('After while')

i = 0
while i < 3:
print('Hello world')
i = i + 1

i = 0
while i < 10:
print('print("Hello world ' + str(i * 9) + '")')
i = i + 1

i = 0
while i < 10:
if i == 4:
print(i)
i = i + 1

print('---컨테이너와 반복문-------------------------------------------------------------------------')
members = ['egoing', 'leezche', 'graphittie']
i = 0
while i < len(members):
print(members[i])
i = i + 1
members = ['egoing', 'leezche', 'graphittie']
for member in members:
print(member)
for item in range(5, 11):
print(item)
input_id = input("아이디를 입력해주세요.\n")
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == input_id:
print('Hello!, ' + member)
import sys
sys.exit()
print('Who are you?')

print('---함수---------------------------------------------------------------------------')
def a3():
print('aaa')
a3()
print('------------------------------------------------------------------------------------------')
def a3():
return 'aaa'
print(a3())
print('------------------------------------------------------------------------------------------')
def a(num):
return 'a' * num
print(a(3))
print('------------------------------------------------------------------------------------------')
input_id = input("아이디를 입력해주세요.\n")
def login(_id):
members = ['egoing', 'k8805', 'leezche']
for member in members:
if member == _id:
return True
return False
if login(input_id):
print('Hello, ' + input_id)
else:
print('Who are you?')
print('---모듈-------------------------------------------------------------------------------')
import math

print(math.ceil(2.9))
print(math.floor(2.9))
print(math.sqrt(16))

print('----모듈이 없을때-----------------------------------------------------------------------------------')
def a():
return 'B'
from egoing import a as z
import k8805 as k

print(z())
print(k.a())

print('---객체제작_객체사용---------------------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2

def add(self):
return self.v1 + self.v2

def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())

print('------객체제작_클래스-----------------------------------------------------------------')
class Cal(object):
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체제작_생성자-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
print(v1, v2)
c1 = Cal(10, 10)
# print(c1.add())
# print(c1.subtract())
c2 = Cal(30, 20)
# print(c2.add())
# print(c2.subtract())
print('------객체제작_인스턴스 변수와 메소드-----------------------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
self.v1 = v1
self.v2 = v2

def add(self):
return self.v1 + self.v2

def subtract(self):
return self.v1 - self.v2
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c2 = Cal(30, 20)
print(c2.add())
print(c2.subtract())
print('------객체와변수_인스턴스 변수의 특성-----------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v

def show(self):
print(self.value)
c1 = C(10)
print(c1.value)
c1.value = 20
print(c1.value)
c1.show()
print('------객체와변수_set/get 메소드--------------------------------------------------------------')
class C(object):
def __init__(self, v):
self.value = v

def show(self):
print(self.value)

def getValue(self):
return self.value

def setValue(self, v):
self.value = v
c1 = C(10)
print(c1.getValue())
c1.setValue(20)
print(c1.getValue())
print('------객체와변수_set/get 메소드를 사용해야하는 이유--------------------------------------------------')
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2

def add(self):
return self.v1 + self.v2

def subtract(self):
return self.v1 - self.v2

def setV1(self, v):
if isinstance(v, int): # v 가 int class의 인스턴스인지 확인
self.v1 = v

def getV1(self):
return self.v1
c1 = Cal(10, 10)
print(c1.add())
print(c1.subtract())
c1.setV1('one')
c1.v2 = 30
print(c1.add())
print(c1.subtract())

print('---------객체와변수_속성 : 실행않됨---------------------------------------------------------------------')
"""
class C(object):
def __init__(self, v):
self.__value = v
def show(self):
print(self.__value)
c1 = C(10)
print(c1.__value)
c1.show()

"""
print('---상속_상속의 문법---------------------------------------------------------------------------')
class Class1(object):
def method1(self): return 'm1'
c1 = Class1()
print(c1, c1.method1())
class Class3(Class1):
def method2(self): return 'm2'
c3 = Class3()
print(c3, c3.method1())
print(c3, c3.method2())

class Class2(object):
def method1(self): return 'm1'
def method2(self): return 'm2'
c2 = Class2()
print(c2, c2.method1())
print(c2, c2.method2())

print('-----상속_상속의 응용-------------------------------------------------------------------')
"""
클래스의 재활용/함수추가 : 타인의 또는 자신의 클래스를 상속하고, 새로운 기능추가, 상속은 변수까지도 이어받음
"""
class Cal(object):
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2

def add(self):
return self.v1 + self.v2

def subtract(self):
return self.v1 - self.v2

def setV1(self, v):
if isinstance(v, int):
self.v1 = v

def getV1(self):
return self.v1


class CalMultiply(Cal):
def multiply(self):
return self.v1 * self.v2


class CalDivide(CalMultiply):
def divide(self):
return self.v1 / self.v2


c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())

print('-------------clss멤버_클래스 매소드--------------------------------------------------------------')
class Cs:
@staticmethod
def static_method():
print("Static method")

@classmethod
def class_method(cls):
print("Class method")

def instance_method(self):
print("Instance method")

i = Cs()
Cs.static_method()
Cs.class_method()
i.instance_method()

print('-------------clss멤버_클래스 변수---------------------------------------------------------------')
class Cs:
count = 0

def __init__(self):
Cs.count = Cs.count + 1

@classmethod
def getCount(cls):
return Cs.count
i1 = Cs()
i2 = Cs()
i3 = Cs()
i4 = Cs()
print(Cs.getCount())

print('-------------clss멤버_clss member의 활용---------------------------------------------------------------')
class Cal(object):
_history = []

def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2

def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result

def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result

def setV1(self, v):
if isinstance(v, int):
self.v1 = v

def getV1(self):
return self.v1

@classmethod
def history(cls):
for item in Cal._history:
print(item)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result
c1 = CalMultiply(10, 10)
print(c1.add())
print(c1.multiply())
c2 = CalDivide(20, 10)
print(c2, c2.add())
print(c2, c2.multiply())
print(c2, c2.divide())
Cal.history()
print('-------------override_형식---------------------------------------------------------------')
class C1:
def m(self):
return 'parent'
class C2(C1):
def m(self):
return super().m() + ' child'
pass
o = C2()
print(o.m())

print('-------------override_override 활용---------------------------------------------------------------')
class Cal(object):
_history = []

def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2

def add(self):
result = self.v1 + self.v2
Cal._history.append("add : %d+%d=%d" % (self.v1, self.v2, result))
return result

def subtract(self):
result = self.v1 - self.v2
Cal._history.append("subtract : %d-%d=%d" % (self.v1, self.v2, result))
return result

def setV1(self, v):
if isinstance(v, int):
self.v1 = v

def getV1(self):
return self.v1

@classmethod
def history(cls):
for item in Cal._history:
print(item)

def info(self):
return "Cal => v1 : %d, v2 : %d" % (self.v1, self.v2)
class CalMultiply(Cal):
def multiply(self):
result = self.v1 * self.v2
Cal._history.append("multiply : %d*%d=%d" % (self.v1, self.v2, result))
return result

def info(self):
return "CalMultiply => %s" % super().info()
class CalDivide(CalMultiply):
def divide(self):
result = self.v1 / self.v2
Cal._history.append("divide : %d/%d=%d" % (self.v1, self.v2, result))
return result

def info(self):
return "CalDivide => %s" % super().info()
c0 = Cal(30, 60)
print(c0.info())
c1 = CalMultiply(10, 10)
print(c1.info())
c2 = CalDivide(20, 10)
print(c2.info())

print('-------------객체와 모듈---------------------------------------------------------------')
import lib
obj = lib.A()
print(obj.a())

print('-------------다중상속_형식,단점---------------------------------------------------------------')
class C1():
def c1_m(self):
print("c1_m")

def m(self):
print("C1 m")
class C2():
def c2_m(self):
print("c2_m")
def m(self):
print("C2 m")

class C3(C2, C1):
def m(self):
print("C3 m")
c = C3()
c.c1_m()
c.c2_m()
c.m()
print(C3.__mro__)

print('-------------다중상속_다중상속의 활용---------------------------------------------------------------')
class CalMultiply():
def multiply(self):
return self.v1 * self.v2
class CalDivide():
def divide(self):
return self.v1 / self.v2
class Cal(CalMultiply, CalDivide): # 왼쪽부모가 우선순위가 높다
def __init__(self, v1, v2):
if isinstance(v1, int):
self.v1 = v1
if isinstance(v2, int):
self.v2 = v2
def add(self):
return self.v1 + self.v2

def subtract(self):
return self.v1 - self.v2
def setV1(self, v):
if isinstance(v, int):
self.v1 = v
def getV1(self):
return self.v1
c = Cal(100, 10)
print(c.add())
print(c.multiply())

print(c.divide())



[참고한 자료]


https://opentutorials.org/course/1750    생활코딩 

Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid

인공지능 구현에 대한 글입니다.
글의 순서는 아래와 같습니다.

================================================
1. multinomial  적용
  ==>합격/불합격이 아닌, A,B,C,D,를 예측하는 개념임
2. Fancy softmax classfication 구현
  ==> 동물들(레이블, 클래스)를 여러특징들(x)로 예측함.
3. Next step
  ==> Multinomial  : 여러개의 classification의 확율을 구함.
4. 참고자료
=================================================

multinomial  적용]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 6 Softmax Classifier

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


x_data = [[1, 2, 1, 1],

          [2, 1, 3, 2],

          [3, 1, 3, 4],

          [4, 1, 5, 5],

          [1, 7, 5, 5],

          [1, 2, 5, 6],

          [1, 6, 6, 6],

          [1, 7, 7, 7]]

y_data = [[0, 0, 1], ==>2

          [0, 0, 1], ==>2

          [0, 0, 1],

          [0, 1, 0], ==>1

          [0, 1, 0],

          [0, 1, 0],

          [1, 0, 0],

          [1, 0, 0]]


X = tf.placeholder("float", [None, 4])

Y = tf.placeholder("float", [None, 3])

nb_classes = 3


W = tf.Variable(tf.random_normal([4, nb_classes]), name='weight')

b = tf.Variable(tf.random_normal([nb_classes]), name='bias')


# tf.nn.softmax computes softmax activations


==> 확율( 1~0 )으로 변환해줌

# softmax = exp(logits) / reduce_sum(exp(logits), dim)

hypothesis = tf.nn.softmax(tf.matmul(X, W) + b)




# Cross entropy cost/loss 

cost = tf.reduce_mean(-tf.reduce_sum(Y * tf.log(hypothesis), axis=1))


D(S,L)=-SIGMA Li log(Si) 함수와   

는 같은 결과를 나타냄


예측이 맞으면  COST는  작아지고, 틀리면 커지도록 구성한 함수임.

상기 함수(상기 2함수는 같은결과를 내고있음 , 같은 함수라고 할수있음)가 그 역할을 충실히 해내고 있음.


optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)

COST 최소화 실행


# Launch graph

with tf.Session() as sess:

    sess.run(tf.global_variables_initializer())


    for step in range(2001):

        sess.run(optimizer, feed_dict={X: x_data, Y: y_data})

        if step % 200 == 0:

            print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}))


    print('--------------')


    # Testing & One-hot encoding

    a = sess.run(hypothesis, feed_dict={X: [[1, 11, 7, 9]]})

    print(a, sess.run(tf.arg_max(a, 1)))

    ==> 예측치 출력(3개) , one hot 값 출력 ==> 어떤 클래스 인지를 결정해서 출력해줌

    ==> 예시)손글씨의 특징을 3개의 x 값으로 표현하고, 예측치를 출력 3개로 하며, 이때 확율이 가장높게나타난 클래스가  해당 손글씨가 됨.

    ==> 예시) 글래스(강아지,고양이...)등의 특징을 여러 x갋으로 표현하고, 예측을 y값으로 예측하고, 그중 확율이 제일 높은수치를 선택해줌, 제일 큰 수치가 해당클래스(고양이, 강아지 등)가 된다.


    print('--------------')


    b = sess.run(hypothesis, feed_dict={X: [[1, 3, 4, 3]]})

    print(b, sess.run(tf.arg_max(b, 1)))


    print('--------------')


    c = sess.run(hypothesis, feed_dict={X: [[1, 1, 0, 1]]})

    print(c, sess.run(tf.arg_max(c, 1)))


    print('--------------')


    all = sess.run(hypothesis, feed_dict={

                   X: [[1, 11, 7, 9], [1, 3, 4, 3], [1, 1, 0, 1]]})

    print(all, sess.run(tf.arg_max(all, 1)))


'''

--------------

[[  1.38904958e-03   9.98601854e-01   9.06129117e-06]] [1]  ==> 전체의 합은 1 임

--------------

[[ 0.93119204  0.06290206  0.0059059 ]] [0]

--------------

[[  1.27327668e-08   3.34112905e-04   9.99665856e-01]] [2]

--------------

[[  1.38904958e-03   9.98601854e-01   9.06129117e-06]

 [  9.31192040e-01   6.29020557e-02   5.90589503e-03]

 [  1.27327668e-08   3.34112905e-04   9.99665856e-01]] [1 0 2]

'''



[ Fancy softmax classfication 구현 ]

import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 6 Softmax Classifier

import tensorflow as tf

import numpy as np

tf.set_random_seed(777)  # for reproducibility


# Predicting animal type based on various features

xy = np.loadtxt('data-04-zoo.csv', delimiter=',', dtype=np.float32)

x_data = xy[:, 0:-1]

y_data = xy[:, [-1]]


print(x_data.shape, y_data.shape)


nb_classes = 7  # 0 ~ 6


X = tf.placeholder(tf.float32, [None, 16])

Y = tf.placeholder(tf.int32, [None, 1])  # 0 ~ 6

Y_one_hot = tf.one_hot(Y, nb_classes)  # one hot

print("one_hot", Y_one_hot)

Y_one_hot = tf.reshape(Y_one_hot, [-1, nb_classes])

# one hot을 하면 rank가 1개 늘어나게 된다, 이를 다시 줄이는 역할을 하는것이 reshape 임 

print("reshape", Y_one_hot)


W = tf.Variable(tf.random_normal([16, nb_classes]), name='weight')

b = tf.Variable(tf.random_normal([nb_classes]), name='bias')


# tf.nn.softmax computes softmax activations

# softmax = exp(logits) / reduce_sum(exp(logits), dim)

logits = tf.matmul(X, W) + b

hypothesis = tf.nn.softmax(logits)


# Cross entropy cost/loss

cost_i = tf.nn.softmax_cross_entropy_with_logits(logits=logits,

                                                 labels=Y_one_hot)

cost = tf.reduce_mean(cost_i)

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)


prediction = tf.argmax(hypothesis, 1)

correct_prediction = tf.equal(prediction, tf.argmax(Y_one_hot, 1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

# Launch graph

with tf.Session() as sess:

    sess.run(tf.global_variables_initializer())


    for step in range(2000):

        sess.run(optimizer, feed_dict={X: x_data, Y: y_data})

        if step % 100 == 0:

            loss, acc = sess.run([cost, accuracy], feed_dict={

                                 X: x_data, Y: y_data})

            print("Step: {:5}\tLoss: {:.3f}\tAcc: {:.2%}".format(

                step, loss, acc))


    # Let's see if we can predict

    pred = sess.run(prediction, feed_dict={X: x_data})

    # y_data: (N,1) = flatten => (N, ) matches pred.shape

    for p, y in zip(pred, y_data.flatten()):

        print("[{}] Prediction: {} True Y: {}".format(p == int(y), p, int(y)))


'''

Step:     0 Loss: 5.106 Acc: 37.62%

Step:   100 Loss: 0.800 Acc: 79.21%

Step:   200 Loss: 0.486 Acc: 88.12%

Step:   300 Loss: 0.349 Acc: 90.10%

Step:   400 Loss: 0.272 Acc: 94.06%

Step:   500 Loss: 0.222 Acc: 95.05%

Step:   600 Loss: 0.187 Acc: 97.03%

Step:   700 Loss: 0.161 Acc: 97.03%

Step:   800 Loss: 0.140 Acc: 97.03%

Step:   900 Loss: 0.124 Acc: 97.03%

Step:  1000 Loss: 0.111 Acc: 97.03%

Step:  1100 Loss: 0.101 Acc: 99.01%

Step:  1200 Loss: 0.092 Acc: 100.00%



[참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/ 

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

http://www.derivative-calculator.net/

http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247    ==> 미분계산/공식

http://matplotlib.org/users/installing.html   ==>matplotlib 설치

https://www.kaggle.com/==> DATA SAMPLE 을 구할수 있는 사이트



Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid

python 문법에 관한 글 입니다.

인공지능 tensor flow 코딩을 위해 python 문법에 대해 정리를 하고자 합니다.

글의 순서는 아래와 같습니다

==========================================

1.Logistics classification 가설함수

  ==> 합격, 불합격을 구분할때 적용함

2. 당뇨병 예측

3. Next step

  ==> Multinomial  : 여러개의 classification의 확율을 구함.

4. 참고자료 

============================================


Logistics classification 가설함수]



import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'


# Lab 5 Logistic Regression Classifier

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


x_data = [[1, 2],

          [2, 3],

          [3, 1],

          [4, 3],

          [5, 3],

          [6, 2]]


x data에 따라 합부(0,1)로 나타는 Y data를 예측함

# 공부한 시간에 따른 합격여부 data


y_data = [[0],

          [0],

          [0],

          [1],

          [1],

          [1]]


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 2])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([2, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')



sigmoid 함수 사용함


 ==> h(x) 가 0~1 사이에 값을 갖게됨.




==> cost(w)은  y값과 h(x)차이가 크면 1에 가깝고, 작으면 0에 가까워지도록 구현함 , 이를 하나의 공식으로 통일시킴.

  -기존의 cost함수형식으로 하게되면  울퉁불퉁한 2차 곡선 형태되고, 경사를 내려오면서 수많은 기울기 0 지점을 지나게 되어, 목표지점에 다다르지 못하게됨

  -따라서 다른 코스트 함수가 필요함, 아래 함수가 그 대안임




# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))

hypothesis = tf.sigmoid(tf.matmul(X, W) + b)


# cost/loss function

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *

                       tf.log(1 - hypothesis))


train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(cost)


# Accuracy computation

# True if hypothesis>0.5 else False

predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)

accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(10001):

        cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})

        if step % 200 == 0:

            print(step, cost_val)


    # Accuracy report

    h, c, a = sess.run([hypothesis, predicted, accuracy],

                       feed_dict={X: x_data, Y: y_data})

    print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)


'''

0 1.73078

200 0.571512

400 0.507414

600 0.471824

800 0.447585

...

9200 0.159066

9400 0.15656

9600 0.154132

9800 0.151778

10000 0.149496


Hypothesis:  [[ 0.03074029]

 [ 0.15884677]

 [ 0.30486736]

 [ 0.78138196]

 [ 0.93957496]

 [ 0.98016882]]

y 예측치


Correct (Y):  [[ 0.]

 [ 0.]

 [ 0.]

 [ 1.]

 [ 1.]

 [ 1.]]

y 예측치와 실제 y값의 차가 0.5 이상이면 0.  이하면 1 

Accuracy:  1.0

예측정확도 100%

'''


'''


[당뇨병 예측]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 5 Logistic Regression Classifier

import tensorflow as tf

import numpy as np

tf.set_random_seed(777)  # for reproducibility


xy = np.loadtxt('data-03-diabetes.csv', delimiter=',', dtype=np.float32)

x_data = xy[:, 0:-1]

y_data = xy[:, [-1]]


print(x_data.shape, y_data.shape)


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 8])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([8, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis using sigmoid: tf.div(1., 1. + tf.exp(tf.matmul(X, W)))

hypothesis = tf.sigmoid(tf.matmul(X, W) + b)


# cost/loss function

cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) *

                       tf.log(1 - hypothesis))


train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(cost)


# Accuracy computation

# True if hypothesis>0.5 else False

predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)

accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))


# Launch graph

with tf.Session() as sess:

    # Initialize TensorFlow variables

    sess.run(tf.global_variables_initializer())


    for step in range(10001):

        cost_val, _ = sess.run([cost, train], feed_dict={X: x_data, Y: y_data})

        if step % 200 == 0:

            print(step, cost_val)


    # Accuracy report

    h, c, a = sess.run([hypothesis, predicted, accuracy],

                       feed_dict={X: x_data, Y: y_data})

    print("\nHypothesis: ", h, "\nCorrect (Y): ", c, "\nAccuracy: ", a)


'''

0 0.82794

200 0.755181

400 0.726355

600 0.705179

800 0.686631

...

9600 0.492056

9800 0.491396

10000 0.490767


...


 [ 1.]

 [ 1.]

 [ 1.]]

Accuracy:  0.762846




[ 참고자료 ]


https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/ 

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

http://www.derivative-calculator.net/

http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247    ==> 미분계산/공식

http://matplotlib.org/users/installing.html   ==>matplotlib 설치

https://www.kaggle.com/==> DATA SAMPLE 을 구할수 있는 사이트

'''





Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid



인공지능 구현에 대한 글입니다.

글의 순서는 아래와 같습니다.


================================================

1. X의 갯수가3개인 경우

  ==> 코딩이 매우 복잡해짐 , 대안으로 "매트릭스_행렬 "을 사용하면 간단해짐.

2. 매트릭스 적용

  ==> DATA 갯수(X 변수의갯수) 이 많아져도 간단하게 코딩이 가능해짐

3. TENSOR FLOW로 파일에서 DATA 읽어오기

  ==> DATA를  프로그램상에 올리면 메모리한계가 있으므로, DATA를 별도의 화일에 저장하고 불러오는것이 유리함

4. DATA QUE로 읽어들임 시간차로 처리해서 시스템 부하를 줄어줌

 ==> 화일이 많거나, 용량이 커지면  TENSORFLOW에서 자동으로 시간차를 두고 처리해주는 시스템 구현이 가능함

5. Next step

  ==> LOGISTICS 알고리즘 : CLASSSIFICATION 중 주요한 알고리즘.

6.참고자료 


=================================================



[ Cost 최소화 알고리즘 구현 소스코드 분석 ]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 4 Multi-variable linear regression

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility

==> x값이 3개

x1_data = [73., 93., 89., 96., 73.]

x2_data = [80., 88., 91., 98., 66.]

x3_data = [75., 93., 90., 100., 70.]


y_data = [152., 185., 180., 196., 142.]


# placeholders for a tensor that will be always fed.

x1 = tf.placeholder(tf.float32)

x2 = tf.placeholder(tf.float32)

x3 = tf.placeholder(tf.float32)


Y = tf.placeholder(tf.float32)


w1 = tf.Variable(tf.random_normal([1]), name='weight1')

w2 = tf.Variable(tf.random_normal([1]), name='weight2')

w3 = tf.Variable(tf.random_normal([1]), name='weight3')

b = tf.Variable(tf.random_normal([1]), name='bias')


hypothesis = x1 * w1 + x2 * w2 + x3 * w3 + b

print(hypothesis)

==> x,w값이 3개(x1,x2,x3 ,w1,w2,w3) 인 가설임

==> 코딩이 매우 복잡해짐 , 대안으로 "매트릭스_행렬 "을 사용하면 간단해짐.

# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize. Need a very small learning rate for this data set

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(2001):

    cost_val, hy_val, _ = sess.run([cost, hypothesis, train],

                                   feed_dict={x1: x1_data, x2: x2_data, x3: x3_data, Y: y_data})

    if step % 10 == 0:

        print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)


'''

0 Cost:  19614.8

Prediction:

 [ 21.69748688  39.10213089  31.82624626  35.14236832  32.55316544]

10 Cost:  14.0682

Prediction:

 [ 145.56100464  187.94958496  178.50236511  194.86721802  146.08096313]


 ...


1990 Cost:  4.9197

Prediction:

 [ 148.15084839  186.88632202  179.6293335   195.81796265  144.46044922]

2000 Cost:  4.89449

Prediction:

 [ 148.15931702  186.8805542   179.63194275  195.81971741  144.45298767]

'''


[ 매트릭스 적용 ]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 4 Multi-variable linear regression

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility



매트릭로 DATA 적용


x_data = [[73., 80., 75.],

          [93., 88., 93.],

          [89., 91., 90.],

          [96., 98., 100.],

          [73., 66., 70.]]

y_data = [[152.],

          [185.],

          [180.],

          [196.],

          [142.]]


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 3])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([3, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


매트릭스 곱샘 코딩

# Hypothesis

hypothesis = tf.matmul(X, W) + b




# Simplified cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(2001):

    cost_val, hy_val, _ = sess.run(

        [cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})

    if step % 10 == 0:

        print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)


'''

0 Cost:  7105.46

Prediction:

 [[ 80.82241058]

 [ 92.26364136]

 [ 93.70250702]

 [ 98.09217834]

 [ 72.51759338]]

10 Cost:  5.89726

Prediction:

 [[ 155.35159302]

 [ 181.85691833]

 [ 181.97254944]

 [ 194.21760559]

 [ 140.85707092]]


...


1990 Cost:  3.18588

Prediction:

 [[ 154.36352539]

 [ 182.94833374]

 [ 181.85189819]

 [ 194.35585022]

 [ 142.03240967]]

2000 Cost:  3.1781

Prediction:

 [[ 154.35881042]

 [ 182.95147705]

 [ 181.85035706]

 [ 194.35533142]

 [ 142.036026  ]]


'''



TENSOR FLOW로 파일에서 DATA 읽어오기 ]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

# Lab 4 Multi-variable linear regression

import tensorflow as tf

import numpy as np

tf.set_random_seed(777)  # for reproducibility


DATA는 CSV 화일에 저장됨, CSV 화일 위치는 PY 화일위치에 있음

xy = np.loadtxt('data-01-test-score.csv', delimiter=',', dtype=np.float32)


슬라이싱방법 숙지필요함

x_data = xy[:, 0:-1]

y_data = xy[:, [-1]]


# Make sure the shape and data are OK

print(x_data.shape, x_data, len(x_data))

print(y_data.shape, y_data)


# placeholders for a tensor that will be always fed.

X = tf.placeholder(tf.float32, shape=[None, 3])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([3, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis

hypothesis = tf.matmul(X, W) + b


# Simplified cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(2001):

    cost_val, hy_val, _ = sess.run(

        [cost, hypothesis, train], feed_dict={X: x_data, Y: y_data})

    if step % 10 == 0:

        print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)


나의 점수 예측

# Ask my score

print("Your score will be ", sess.run(

    hypothesis, feed_dict={X: [[100, 70, 101]]}))




print("Other scores will be ", sess.run(hypothesis,

                                        feed_dict={X: [[60, 70, 110], [90, 100, 80]]}))


'''


타인의 점수 예측

Your score will be  [[ 181.73277283]]

Other scores will be  [[ 145.86265564]

 [ 187.23129272]]


'''


DATA QUE로 읽어들임 시간차로 처리해서 시스템 부하를 줄어줌 ]



import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


화일 하나를 위한 QUE를 만듦

filename_queue = tf.train.string_input_producer(

    ['data-01-test-score.csv'], shuffle=False, name='filename_queue')


화일 리더 정의

reader = tf.TextLineReader()

key, value = reader.read(filename_queue)


# Default values, in case of empty columns. Also specifies the type of the

# decoded result.


CSV로 디코딩

record_defaults = [[0.], [0.], [0.], [0.]]

xy = tf.decode_csv(value, record_defaults=record_defaults)



불러올때마다 10 개씩 불러옴 (BATCH_SIZE =10 )

# collect batches of csv in

train_x_batch, train_y_batch = \

    tf.train.batch([xy[0:-1], xy[-1:]], batch_size=10)


# placeholders for a tensor that will be always fed.


X 3개 , Y 1개 데이터 숫자를 맞추어줌


X = tf.placeholder(tf.float32, shape=[None, 3])

Y = tf.placeholder(tf.float32, shape=[None, 1])


W = tf.Variable(tf.random_normal([3, 1]), name='weight')

b = tf.Variable(tf.random_normal([1]), name='bias')


# Hypothesis

hypothesis = tf.matmul(X, W) + b


# Simplified cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize

optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-5)

train = optimizer.minimize(cost)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


# Start populating the filename queue.

coord = tf.train.Coordinator()

threads = tf.train.start_queue_runners(sess=sess, coord=coord)


for step in range(2001):


x_batch , y_batch 값을  train feed_dict로 넘겨줌

    x_batch, y_batch = sess.run([train_x_batch, train_y_batch])

    cost_val, hy_val, _ = sess.run(

        [cost, hypothesis, train], feed_dict={X: x_batch, Y: y_batch})

    if step % 10 == 0:

        print(step, "Cost: ", cost_val, "\nPrediction:\n", hy_val)


coord.request_stop()

coord.join(threads)


# Ask my score

print("Your score will be ",

      sess.run(hypothesis, feed_dict={X: [[100, 70, 101]]}))


print("Other scores will be ",

      sess.run(hypothesis, feed_dict={X: [[60, 70, 110], [90, 100, 80]]}))


'''

Your score will be  [[ 177.78144836]]

Other scores will be  [[ 141.10997009]

 [ 191.17378235]]


'''




[참고자료 ]


https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

http://www.derivative-calculator.net/

http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247    ==> 미분계산/공식

http://matplotlib.org/users/installing.html   ==>matplotlib 설치




Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid



인공지능 구현에 대한 글입니다.

글의 순서는 아래와 같습니다.


================================================

1. Cost 최소화 알고리즘 구현 소스코드 분석

2. Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용)

3. Optimizer 적용 테스트

 - 간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 optimizer로 코딩함

4. Optimizer 적용 과 수작업(미분계산) 수치 비교

 - 경사도 수치를 중간에 변형할수 있도록 코딩함 (gvs) ==> 거의 같음

5. Next step

  ==> multivariable linear regresstion : 하나이상의 변수적용

6. 참고자료

=================================================


[ Cost 최소화 알고리즘 구현 소스코드 분석 ]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'


# Lab 3 Minimizing Cost

import tensorflow as tf

import matplotlib.pyplot as plt

==> matplotlib를 사용키 위해서는 별도로 설치를 해야함

==> http://matplotlib.org/users/installing.html

==> 그래프를 그려주는 기능임.


tf.set_random_seed(777)  # for reproducibility


X = [1, 2, 3]

Y = [1, 2, 3]


W = tf.placeholder(tf.float32)


# Our hypothesis for linear model X * W

hypothesis = X * W


# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Launch the graph in a session.

sess = tf.Session()


# Variables for plotting cost function

W_history = []

cost_history = []

==> 그래프를 그리기 위해 배열할당


for i in range(-30, 50):

    curr_W = i * 0.1

    curr_cost = sess.run(cost, feed_dict={W: curr_W})

    W_history.append(curr_W)

    cost_history.append(curr_cost)

# Show the cost function

plt.plot(W_history, cost_history)

plt.show()



Cost Minimize 코딩 작성 ( cost 를 미분해서 W변화량 적용) ]


import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'


# Lab 3 Minimizing Cost

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


x_data = [1, 2, 3]

y_data = [1, 2, 3]


# Try to find values for W and b to compute y_data = W * x_data + b

# We know that W should be 1 and b should be 0

# But let's use TensorFlow to figure it out

W = tf.Variable(tf.random_normal([1]), name='weight')


X = tf.placeholder(tf.float32)

Y = tf.placeholder(tf.float32)


# Our hypothesis for linear model X * W

hypothesis = X * W


# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))




W값의  descent 간격을 구하는 과정을 미분방정식으로 직접(수작업) 코딩하는 내용입니다( 아래그림은 미분적용 과정입니다)


 



# Minimize: Gradient Descent using derivative: W -= learning_rate * derivative

learning_rate = 0.1

gradient = tf.reduce_mean((W * X - Y) * X)

descent = W - learning_rate * gradient

update = W.assign(descent)


간단한 미분은 상기과 같이 직접코딩이 가능하겠으나, 복잡해지면 통상적으로 아래와 같이 코딩을 하는것이 일반적임(텐서플로우가 자동계산을 해줌)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(21):

    sess.run(update, feed_dict={X: x_data, Y: y_data})

    print(step, sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))


'''

0 1.93919 [ 1.64462376]

1 0.551591 [ 1.34379935]

2 0.156897 [ 1.18335962]

3 0.0446285 [ 1.09779179]

4 0.0126943 [ 1.05215561]

5 0.00361082 [ 1.0278163]

6 0.00102708 [ 1.01483536]

7 0.000292144 [ 1.00791216]

8 8.30968e-05 [ 1.00421977]

9 2.36361e-05 [ 1.00225055]

10 6.72385e-06 [ 1.00120032]

11 1.91239e-06 [ 1.00064015]

12 5.43968e-07 [ 1.00034142]

13 1.54591e-07 [ 1.00018203]

14 4.39416e-08 [ 1.00009704]

15 1.24913e-08 [ 1.00005174]

16 3.5322e-09 [ 1.00002754]

17 9.99824e-10 [ 1.00001466]

18 2.88878e-10 [ 1.00000787]

19 8.02487e-11 [ 1.00000417]

20 2.34053e-11 [ 1.00000226]

'''


[ Optimizer 적용 테스트 ]

import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # Lab 3 Minimizing Cost import tensorflow as tf tf.set_random_seed(777) # for reproducibility # tf Graph Input X = [1, 2, 3] Y = [1, 2, 3] # Set wrong model weights W = tf.Variable(5.0) ==> w 값을 터무니없는 값을 주고, 적정한 값을 찾아가는지 테스트함 # Linear model hypothesis = X * W # cost/loss function cost = tf.reduce_mean(tf.square(hypothesis - Y)) # Minimize: Gradient Descent Magic optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1) train = optimizer.minimize(cost) # Launch the graph in a session. sess = tf.Session() # Initializes global variables in the graph. sess.run(tf.global_variables_initializer()) for step in range(100): print(step, sess.run(W)) sess.run(train) ''' 0 5.0 1 1.26667 2 1.01778 3 1.00119 4 1.00008 ... 96 1.0 97 1.0 98 1.0 99 1.0 '''



[ Optimizer 적용 과 수작업(미분계산) 수치 비교]


 import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'


# Lab 3 Minimizing Cost

# This is optional

import tensorflow as tf

tf.set_random_seed(777)  # for reproducibility


# tf Graph Input

X = [1, 2, 3]

Y = [1, 2, 3]


# Set wrong model weights

W = tf.Variable(5.)


# Linear model

hypothesis = X * W


# Manual gradient

gradient = tf.reduce_mean((W * X - Y) * X) * 2


# cost/loss function

cost = tf.reduce_mean(tf.square(hypothesis - Y))


# Minimize: Gradient Descent Magic

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)

train = optimizer.minimize(cost)


# Get gradients

gvs = optimizer.compute_gradients(cost, [W])

# Optional: modify gradient if necessary

# gvs = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gvs]

==> gvc 수치를 필요에 따라서 변형적용이 가능함


# Apply gradients

apply_gradients = optimizer.apply_gradients(gvs)


# Launch the graph in a session.

sess = tf.Session()

# Initializes global variables in the graph.

sess.run(tf.global_variables_initializer())


for step in range(100):

    print(step, sess.run([gradient, W, gvs]))

    sess.run(apply_gradients)

    # Same as sess.run(train)



'''

# Apply gradients

0 [37.333332, 5.0, [(37.333336, 5.0)]]

1 [33.848888, 4.6266665, [(33.848888, 4.6266665)]]

2 [30.689657, 4.2881775, [(30.689657, 4.2881775)]]

3 [27.825287, 3.9812808, [(27.825287, 3.9812808)]]

4 [25.228262, 3.703028, [(25.228264, 3.703028)]]

...

96 [0.0030694802, 1.0003289, [(0.0030694804, 1.0003289)]]

97 [0.0027837753, 1.0002983, [(0.0027837753, 1.0002983)]]

98 [0.0025234222, 1.0002704, [(0.0025234222, 1.0002704)]]

99 [0.0022875469, 1.0002451, [(0.0022875469, 1.0002451)]]

'''

[ 참고자료 ]

https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/ 

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

http://www.derivative-calculator.net/

http://terms.naver.com/entry.nhn?docId=3350391&cid=58247&categoryId=58247    ==> 미분계산/공식

http://matplotlib.org/users/installing.html   ==>matplotlib 설치



Comment +0

 daviduino.co.kr

techtogetworld.com

 david201207.blog.me

cafe.naver.com/3dpservicedavid


인공지능 구현에 대한 글입니다.

글의 순서는 아래와 같습니다.


================================================

1. Tensorflow  Linear regression 소스코드 설명

2. placeholders 적용

  ==> data x,y를 미리 주지않고, 빈공간으로 그래프를 구성하고, 나중에 실행단계에서 숫자를 입력하는 방식임

  ==> 모델을 만들어 놓고, 나중에 data를 넘겨줄수 있음. 

3. Next step

  ==> cost 최소화 알고리즘 상세분석

4. 참고자료

=================================================

[Tensorflow Linear regression 소스코드 설명 ]


1. 라이브러리 IMPORT import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # Lab 2 Linear Regression import tensorflow as tf tf.set_random_seed(777) # for reproducibility

2. DATA 할당


# X and Y data x_train = [1, 2, 3] y_train = [1, 2, 3] # Try to find values for W and b to compute y_data = x_data * W + b # We know that W should be 1 and b should be 0 # But let TensorFlow figure it out


3. 변수선언
W = tf.Variable(tf.random_normal([1]), name='weight') b = tf.Variable(tf.random_normal([1]), name='bias')

# Our hypothesis XW+b


4. 가설 : X에 값에 따라 선형으로 변화하는 Y값의 관계를 직선의 공식으로 표현하고, 이를 파이선 문법으로 작성함



hypothesis = x_train * W + b



5. COST 함수 : 가설값과 실제 Y값의 차이를 최소화 하는 W를 찾는 함수



# cost/loss function cost = tf.reduce_mean(tf.square(hypothesis - y_train))



6. Gradient Descent : W 값을 기울기(미분값)*알파 값만큼 이동시키면서, COST 최소값을 찾아가는 함수

# Minimize optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01) train = optimizer.minimize(cost) # Launch the graph in a session.


7. SESSION 선언 : 텐서플로우 노드를 실행하기 위해서는 SESSION선언후 실행이 가능함

sess = tf.Session() # Initializes global variables in the graph.


8. 실행

sess.run(tf.global_variables_initializer())

- 변수를 실행하기 위해서는 글로벌 로 초기화를 해주어야 함. # Fit the line for step in range(2001): sess.run(train) ==> Train을 실행하면, 하위단의 cost , hypathsis , w ,b 등이 따라서 실행이 되는 구조임 if step % 20 == 0: print(step, sess.run(cost), sess.run(W), sess.run(b)) # Learns best fit W:[ 1.], b:[ 0.]


9. 실행결과

- STEP이 진행될수록 Cost값 ( 실제값과 가설값의 차이 ) 가 작아지는 방향으로 W 값과 , B 값을 을 찾아가고 있음을 알수있음

- 여기서는 w값이 1에 수렴하고 , b는 0에 수렴하는것으로 보여집니다.

- 즉

X 1,2,3 / Y 1,2,3 을 가장 잘 나타내는 식은

H(X)=1*X+0 임을 알수있습니다. 0를 지나는 기울기 1인이 직선이 DATA를 나타내는 최적의 선 임을 알수있음


''' 0 2.82329 [ 2.12867713] [-0.85235667] 20 0.190351 [ 1.53392804] [-1.05059612] 40 0.151357 [ 1.45725465] [-1.02391243] ... 1920 1.77484e-05 [ 1.00489295] [-0.01112291] 1940 1.61197e-05 [ 1.00466311] [-0.01060018] 1960 1.46397e-05 [ 1.004444] [-0.01010205] 1980 1.32962e-05 [ 1.00423515] [-0.00962736] 2000 1.20761e-05 [ 1.00403607] [-0.00917497]


'''



[ placeholders 적용 ]

import os os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # Lab 2 Linear Regression import tensorflow as tf tf.set_random_seed(777) # for reproducibility # Try to find values for W and b to compute y_data = W * x_data + b # We know that W should be 1 and b should be 0 # But let's use TensorFlow to figure it out W = tf.Variable(tf.random_normal([1]), name='weight') b = tf.Variable(tf.random_normal([1]), name='bias') # Now we can use X and Y in place of x_data and y_data # # placeholders for a tensor that will be always fed using feed_dict # See http://stackoverflow.com/questions/36693740/
X = tf.placeholder(tf.float32, shape=[None]) Y = tf.placeholder(tf.float32, shape=[None]) ==> data x,y를 미리 주지않고, 빈공간으로 그래프를 구성하고, 나중에 실행단계에서 숫자를 입력하는 방식임
# Our hypothesis XW+b hypothesis = X * W + b # cost/loss function cost = tf.reduce_mean(tf.square(hypothesis - Y)) # Minimize optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01) train = optimizer.minimize(cost) # Launch the graph in a session. sess = tf.Session() # Initializes global variables in the graph. sess.run(tf.global_variables_initializer()) # Fit the line for step in range(2001): cost_val, W_val, b_val, _ = \ sess.run([cost, W, b, train], feed_dict={X: [1, 2, 3], Y: [1, 2, 3]})
==> cost , w, b 실행을 한번에 명령을 줄수있음
==> cost data x,y를 여기서 주어줌 if step % 20 == 0: print(step, cost_val, W_val, b_val) # Learns best fit W:[ 1.], b:[ 0] ''' ... 1980 1.32962e-05 [ 1.00423515] [-0.00962736] 2000 1.20761e-05 [ 1.00403607] [-0.00917497] ''' # Testing our model print(sess.run(hypothesis, feed_dict={X: [5]})) print(sess.run(hypothesis, feed_dict={X: [2.5]})) print(sess.run(hypothesis, feed_dict={X: [1.5, 3.5]})) ''' [ 5.0110054] [ 2.50091505] [ 1.49687922 3.50495124] ''' # Fit the line with new training data for step in range(2001): cost_val, W_val, b_val, _ = \ sess.run([cost, W, b, train], feed_dict={X: [1, 2, 3, 4, 5], Y: [2.1, 3.1, 4.1, 5.1, 6.1]})
==> shape를 none으로 했기때문에,data 갯수는 필요한 만큼 줄수있음 if step % 20 == 0: print(step, cost_val, W_val, b_val) # Testing our model print(sess.run(hypothesis, feed_dict={X: [5]})) print(sess.run(hypothesis, feed_dict={X: [2.5]})) print(sess.run(hypothesis, feed_dict={X: [1.5, 3.5]}))

==> 학습이 완료된 hypothis 함수에 X값을 주고, Y값을 예측하는 과정임
''' 1960 3.32396e-07 [ 1.00037301] [ 1.09865296] 1980 2.90429e-07 [ 1.00034881] [ 1.09874094] 2000 2.5373e-07 [ 1.00032604] [ 1.09882331] [ 6.10045338] [ 3.59963846] [ 2.59931231 4.59996414]
"""

[참고자료 ]


https://www.inflearn.com/course/기본적인-머신러닝-딥러닝-강좌/ 

http://agiantmind.tistory.com/176 

https://www.tensorflow.org/install/

https://github.com/hunkim/deeplearningzerotoall

Comment +0