Description

Assignment: You can complete one of the two parts to this assignment, or both. If you do both parts, you can obtain extra points for this assignment. You are expected to complete at least one part successfully.

Data File: CreditHistory_Clean.xlsx

Part 1: Create a SAS EM project named “Week 5 Homework”. In that project read this data file for this assignment. Import the data, ensuring that all attributes have the proper metadata described in the data dictionary. In this case, the target is ‘good_bad’, a binary target.

These data do no contain missing values or outliers. There is no need to “clean” these data. Use ‘gini’ as the split criterion rather than the default ‘ProbChisq’. Gini is the default used in python.

Use 4-fold cross validation to determine the best FNN (Forward Neural Network) for configurations (3), (11), (5,4), (6,5), (7,6), and (8,7). For each network configuration, calculate recall, accuracy, precision and F1. Use these metrics to select the best configuration for your FNN, based upon the HP Neural Network.

After you have selected the best FNN, evaluate your model using a 70/30 training/validation split. Calculate the same metrics for the validation data.

Part 1: SAS EM Solution Upload (all screen shots must be readable)
1. A screen shot of your project diagram
2. A screen shot or listing of ALL SAS code used in your diagram. 3. A table of the metrics for each of the 4 cross-validation folds
4. Describe which model you selected and why.
5. A table of the same metric for the 70/30 test of your selected model.
Part 2: Do the same assignment as Part 1 using Python.

Use one-hot encoding for the nominal attributes, but do not bother to scale the interval attributes. Instead of running depths described for Part 1, use configurations (3), (11), (5,4), (6,5), (7,6), and (8,7). It’s easier in python to vary these networks, and with a small dataset it runs quickly.

Also use the parameter drop=False with the ReplaceImputeEncode() method. This was just added for trees, and specifies not to drop the last column for a nominal feature.

Prepare a report containing:

1. A listing of your python code.
2. A table of the metrics (recall, accuracy, precision and F1) calculated for each of your 4 cross-validation folds.
3. Describe which model you selected from Cross-Validation, and why.
4. A table of the metrics (recall, accuracy, precision and F1) for the 70/30 split using your selected model.