ECE 473 : Introduction to Artificial Intelligence

Assignment 3

Instructions on programming assignments

You are asked to modify the code in hw3 submission.py between

############################################################

############################################################

# BEGIN YOUR CODE

# HINTS OR INSTRUCTIONS

pass ;

# END YOUR CODE

############################################################

############################################################

• Please use Python3 to do all assignments in the course.

• You are allowed to make your own helper functions outside the skeleton functions.

• Do not change other than hw3 submission.py. You are only allowed to use numpy package to compute

and construct functions. Do not use scikit-learn, pytorch, or tensorflow etc to implement your

functions.

• You can test your code with test functions in test.py. For your reference, you can compare your own

classifiers with the ones in sklearn package that is implemented in test.py.

• Homework will be graded based on the provided dataset and may be additionally tested on datasets

that are not released to students.

• You need to get no more than 5% difference of accuracy from the reference to get full score.

• Only submit hw3 submission.py

1

Problem 1: Gaussian Naive Bayes Classifier

Calculation of Bayes Theorem can be simplified by making some assumptions, such as independence between

all input variables. Although a dramatic and unrealistic assumption, this has the effect of making the

calculations of the conditional probability tractable and results in an effective classification model referred

to as Naive Bayes.

The Naive Bayes algorithm has proven effective and therefore is popular for text classification tasks. The

words in a document may be encoded as binary (word present), or count (word occurrence) input vectors

and binary, multinomial, or Gaussian probability distributions used.

In this problem, you will implement Gaussian Naive Bayes Classifier and test it on two datasets:

1. breast cancer dataset

2. digits dataset

Digits dataset consist of 1797 samples with 10 classes. Each sample has 64 features, which is originally 8×8

image. Your task for both dataset is to predict correct class for the test samples.

(Note: When you compute Bayes rule, denominator cannot be 0. In other words, a value inside log cannot

be equal or less than 0.)

Tasks in Problem 1:

1. Implement function Naive Bayes.mean

2. Implement function Naive Bayes.std

3. Implement function Naive Bayes.gen by class

4. Implement function Naive Bayes.calc gaussian dist

5. Implement function Naive Bayes.predict

2

Problem 2: Two-Layered Neural Network

Based on the structure of Logistic regression of Homework 2, you will extend the structure to two-layers

neural network in this problem. Note that the counting index of “layers” starts with the first hidden layer

up to the output layer. Thus, as illustrated in Fig 1, two-layered neural network has one hidden layer with

a certain number of hidden units. The number of hidden units can vary considering various aspects such as

how complex the input feature is or how deep the neural network is.

Figure 1: Illustration of two-layered neural network.

In this homework, you will implement a two-layered neural network with a hidden layer with 64 units

and ReLU activation before fed forward to the output layer. Whereas for the output layer, you will use

sigmoid function instead.

Denote x ∈ R

D as an input data with D dimensions and y ∈ {0, 1} as the corresponding true binary

label. We denote f(x; θ) : R

D → R as a two-layered neural network that we want to train. The goal is to

find f that minimizes the loss L in Eq.(1).

L(f(x), y) = LBCE(f(x), y) + λkθk

2

2

. (1)

This loss L consist of two terms: 1) binary cross entropy loss (LBCE) and 2) regularization. The first term

LBCE is identical to the logistic regression in hw2 as in Eq.(2). The second term is the regularization with

the weight λ which is for better generalization and to prevent the model from over-fitting to the training

data. Note that θ includes all trainable parameters including weights and biases in all layers.

To update the parameters with back propagation in the neural network, you will use mini-batch stochastic

gradient descent method as in Homework 2.

LBCE(f(x), y) = E(x,y)

h

y log(f(x)) + (1 − y) log(1 − f(x))i

. (2)

Note: hyperparameters are provided as follows. Do not change the hyperparameter setting.

• learning rate = 10−3

,

• batch size = 64,

• iterations number = 2000,

• regularization hyperparameter λ in Eq. (1) (i.e., reg in hw3 submission.py) = 10−3

.

In this problem, you will test the neural network on breast cancer dataset.

Tasks in Problem 2:

1. Implement function Neural Network.activation

2. Implement function Neural Network.sigmoid

3. Implement function Neural Network.loss

3

4. Implement function Neural Network.fit

5. Implement function Neural Network.predict

4

Sale!

Uncategorized

# Artificial Intelligence Assignment 3

$30.00

ECE 473 : Introduction to Artificial Intelligence

Assignment 3

Instructions on programming assignments

You are asked to modify the code in hw3 submission.py between

############################################################

############################################################

# BEGIN YOUR CODE

# HINTS OR INSTRUCTIONS

pass ;

# END YOUR CODE

############################################################

############################################################

• Please use Python3 to do all assignments in the course.