Description

Instructions
Use Rmarkdown to create and submit a single html or pdf with your answers to question 1-2 along with supporting evidence in the form of tables and graphs.
All tables and graphs should be neatly labeled and appear polished.
Question 1 and 2 ask you to use R. You should submit your code for each problem as ps4_q1.R and ps4_q2_X.R .
You should submit a single compressed archive ( .zip ) which contains the following files:
ps4.pdf or ps4.html ps4.Rmd ps4_q1.R ps4_q2_funcs.R , ps4_q2a.R , ps4_q2b.R , ps4_q2c.R run_ps4_q2b.pbs , run_ps4_q2c.pbs ps4_q2b.Rout , ps4_q2c-X.Rout (X = 1, 2, 4). ps4_q3.sas , ps4_q3c.csv , ps4_q3d.csv
All files should be executable without errors.
All files read, sourced, or referred to within scripts should be assumed to be in the same working directory ( ./ ).
Your code should be clearly written and it should be possible to assess it by reading it. Use appropriate variable names and comments. Your style will be graded using the style rubric (./StyleRubric.html) [15 points].
Question 1 [20 points]
Use the Lahman baseball data previously seen in the SQL notes
Write an SQL query to construct a table showing the all-time leader in hits (“H” from the “batting” table) for each birth country (“birthCountry” in the “master” table). An all-time leader is the player (“playerID”) with the most total hits across all rows (e.g. seasons/stints). Limit your table to players/countries with at least 200 hits and order the table by descending number of hits. Create a nicely formatted table with the following columns as your final output: Player (nameFirst nameLast), Debut (debut), Country of Birth (birthCountry), Hits (H).
Question 2 [40 points]
In this question you will modify your answer to Problem Set 3, Question 2 (PS3 Q2) to practice parallel, asynchronous, and batch computing. Copy the functions from part a and c of PS3 Q2 to a new file ps4_q2_funcs.R
In each of the parts below, let β∈ℝ100 be defined so that
,
i 0
a. Write an R script ps4_q2a.R that sources ps4_q2_funcs.R , and then uses mclapply to run parallel simulations for ρ∈ {.25i}3i=−3.
Let σ = 1 and use 10,000 Monte Carlo replications. Reorganize the results into a long data frame results_q4a with columns: “rho”, “sigma”, “metric”, “method”, “est”, and “se”. “Metric” should
contain the assessment measure: FWER, FDR, Sensitivity, or Specificity and “method” the multiple comparison method used. The columns “est” and “se” should contain the Monte Carlo estimate and its standard error, respectively.
b. Use your script from part a as the basis for a new script ps4_q2b.R . Setup a 4 core cluster using doParallel and then use nested foreach loops to run simulations for ρ∈ {.25i}3i=−3 and
σ = {.25, .5, 1}. Reshape the results as before into results_q4b saved to a file
results_q4b.RData . Use a PBS file to run this script on the Flux cluster.
c. Modify your script from part a to create ps4_q2c.R which reads the following arguments from the command line: sigma , mc_rep , and n_cores . Also modify the script to use the futures package for parallelism. Use a PBS file to run this script as a job array for σ = {.25, .5, 1}. Hint: see the answer at this page (https://stackoverflow.com/questions/12722095/how-do-i-use-floating-point-division-inbash) for how to convert $PBS_ARRAYID to sigma .
Question 3 [25 points]
a. Put the data into a folder ./data and then follow the instructions to read this data into SAS.
b. Use one or more data steps to reduce the data set to those rows with “MRI” in the ‘hcpcs_description’ field and where ‘hcpcs_code’ starts with a 7.
c. Use proc means or proc summary (as needed) to determine the MRI procedures with the highest volume, highest total payment, and highest average payment among the procedures represented here. d. Repeat part b-c using PROC SQL.
e. Export the results from “c” and “d” to csv and verify that they match. You do not need to produce a nice table within your solution document.