Data Wrangling and Visualization Certificate Program (DWV 101)

In this module, you will learn how to install R and RStudio. You will also learn how to make the best use of the Quarto within RStudio to enhance reproducible publication. RStudio is an IDE (Integrated Development Environment) that makes learning R much easier. In Quarto, you can run not only R codes but also Python codes or codes for other programming languages. With this preparation done, you will learn what you can do with R. While the program will teach you Base R, it will focus on the Tidyverse approach of data science with the use of the Tidyverse package. In base R, one had to use several parentheses together, adding a code inside the parentheses and inside yet another parenthesis, making coding complex. This makes your R code hard to read and understand. In modern R, you can zoom on data, using the pipe operator (%>% or |>), which allows information to move from top to bottom through a pipe. That is, you first start with data, then pipe the data into the next steps -- to tidy, transform, or visualize data, resulting in a chain of clear and readable operations. This approach simplifies coding, making it more intuitive and easier to debug. Each step in the chain performs a specific task, allowing you to focus on one operation at a time while maintaining a logical flow.

Upon successful completion of this module, you will be able to:

1. Install R and RStudio.
2. Describe the layout and menus of RStudio.
3. Compare and contrast different types of codebooks in RStudio - R Scripts, Rmd, and Quarto.
4. Be familiar with Quarto Markdown file.
5. Explain typical workflow for a project in RStudio.
6. Install R packages and load them up.
7. Start, Run codes, organize the codes, and save the qmd file, and render it to html, pdf, and Word files.
8. Explain how reproducible report with literate coding enhances efficiency, ethics, transparency, and credibility.
9. Describe the capability of Quarto in R.
10. Explain how to use online resources provided by the R community.

The goal of this module is to introduce an overview of the world of data science in R and get you ready for the rest of the modules. 

Specifically, you will learn the universe of R and data science in general in this module.  As R spreads to the academic and research community, more and more college students are learning statistics with R. R was initially developed as a statistical tool.  What else can we do with R? What do data scientists do with R?

For starters, we can create a chart any way we want with ggplot2. You can create and update Word, HTML, PPT, and PDF files right from the Quarto file. You can animate your charts and make your charts interactive. You can create a website and a dashboard with shiny and shiny Dashboard.  You can build machine learning models with Caret or tidy models. You can run Python in R with the reticulate package. Some packages help you create charts with a menu-driven approach (e.g., ggThemeAssist and esquisse).

In this module, you will learn R's capabilities and resources that you can use to learn the skills. There is no way you can learn everything in this short course. Thus, we will focus on some fundamental topics, while you will be led to resources for advanced topics you can tackle in the future.

Upon successful completion of this module, you will be able to:

1. Describe the concept of the Tidyverse way of coding.
2. Explain the reproducibility crisis in research and the role of Quarto.
3. Create appealing documents with Quarto.
4. Be able to create a website composed of a variety of documents.
5. Describe the capability of the Tidyverse package in R.
6. List the Pros and Cons of R and Python for data science.
7. Be familiar with the debate about R vs. Python. 
8. Set your goal and commit to study allocated time continuously during the duration of the course.

The ggplot 2 is a plotting package that provides helpful commands to create complex plots from data. It provides a more programmatic interface for specifying what variables to plot, how they are displayed, and general visual properties. Therefore, we only need minimal changes if the underlying data changes or if we decide to change from a bar plot to a scatterplot. This helps in creating publication-quality plots with minimal amounts of adjustments and tweaking. Therefore, it is not surprising that ggplot2 is included in Tidyverse.

In this module, you will be introduced to concepts of the grammar of graphics when visualizing data with the ggplot2 package. Every ggplot starts with defining data and the aesthetics of variables, which is the foundation. Then, you add a geometry layer over the foundation. The two are the basics of any plot. The rest of the layers are optional. You will also be introduced to how to create charts when there is only one (or more) continuous or categorical variable(s) in your data set. Furthermore, you will also be introduced to plot charts after you select and filter the variables you want to plot in your data set. Finally, we will also learn other ways of visualizing data with Tableau-like interface using {GwalkR} package and highly-customizable tables using {gt} package. 

Upon successful completion of this module, you will be able to:

1. Explain the concept of the grammar of graphics when visualizing data with the ggplot2 package.
2. Be familiar with various types of charts.
3. Visualize counts, proportions, and geospatial data.
4. Select appropriate charts based on strategic considerations (e.g., the characteristics of the data and audience).
5. Create a chart that involves one or two variables with either categorical or continous data. 
6. Create a chart by adding a categorical moderator (3rd variable) to the chart involving two variables. 
7. Create correlation charts.
8. Read charts and generate insights.
9. Visualize data with a Tableau-like interface.
10. Visualize data with highly customizable tables.

In this module, you will be introduced to advanced topics of the grammar of graphics and ggplot2 extensions such as {patchwork} and {gganimate}. You will also observe how data scientists use ggplot2 to customize charts for better communication, including live screencast demonstrations of clever and creative data visualizations. In addition, you will practice how to customize and style your tables with {gt} and its extension, {gtExtra}. Advanced users or motivated beginners are encouraged to tackle the advanced visualization codebook with interactive, animated, or geospatial charts. 

 Upon successful completion of this module, you will be able to:

1. Customize correlation charts.
2. Customize tables further for publication quality.
3. Create a chart that involves four or five variables by adding layers of geometry.
4. Modify charts for storytelling by customizing axes, labels, coordinates, themes, etc.
5. Explain when to use interactive or animated charts or dashboards.
6. Visualize geospatial data on a map.
7. Read charts and generate insights.

Understanding the concept of Data Wrangling is important because many professionals inside technology industries have to face different data types and also deal with the various sources of data. In this module, you will be introduced to Data Wrangling. In a broad sense, data wrangling includes (1) importing data, (2) tidying data, and (3) transforming data. After the wrangling process, you can proceed with visualizing data and modeling. 

In this module, we will be focusing on the last two stages of the wrangling process -- tidying and transforming data. You will also be introduced to a modern data type called, tibbles and learn how tibbles are different from data frames. One of the first steps in data wrangling in the Tidyverse ecosystem is to reshape the data to be tidy using "tidyr" package. You will learn the concept of tidy data, and how reshaping the data helps you visualize the data. Once you make your data tidy, you will want to transform your data with the help of "dplyr" package. Learn some key functions from dplyr to transform the data. Once wrangling is done, you pipe the wrangled data into ggplot() to do all the necessary mapping and geometry as well as other optional layers of coordinates and themes, according to the grammar of graphics. 

Upon successful completion of this module, you will be able to:

1. Describe the concept of Data Wrangling.
2. Describe how Tibbles are different from data frames.
3. Explain how to convert wide or long data to "Tidy" data.
4. Explain how to merge relational data sets using joins.
5. Explain how to use grouped mutates and filter together.
6. Be familiar with key dplyr verbs and use them to transform data.
7. Create a new variable with mutate() and case_when().
8. Use the pipe operator to shape the data to prepare for analysis and visualization.

Continuing from M05, you will expand your horizons by going deeper into the topics with various approaches. 

First, you will learn how data scientists wrangle and visualize data for their projects by watching demonstrations. We will also demonstrate how to combine multiple data sets collected from consumer panels for an actual project. In addition, you will learn how to create a revealjs presentation slides and style them using CSS. Finally, you will be exposed to more advanced topics and associated functions such as recode(), across(), case_when(), rownames_to_column(), distinct(), rowwise(), and c_across().

Upon successful completion of this module, you will be able to:

1. Use basic wrangling verbs comfortably. 
2. Articulate the kinds of wrangling desired for a situation. 
3. Use RStudio help and online search to learn how to use certain functions. 
4. Join data when there are no unique keys. 
5. Explain how to use across() in various situations. 
6. Do row-wise operations with rowwise(). 
7. Create revealjs presentation slides and style them beautifully. 

Data types (numeric, character, logical, factor, dates) are building blocks of data structures (vectors, matrices, data frames, and lists). Most of the time, you are likely to work with data frames or Tibbles in the Tidyverse framework, which is a spreadsheet with rows of observations and columns of features or variables. Matrices are similar to data frames in that they consist of rows and columns. A major difference is that while matrices are composed of variables that are of the same data type, data frames can have a mix of any type of data. You can perform various operations on data to filter and view parts of data or to create new variables. Knowing the differences between the data types and structures will provide you with the basic knowledge needed to manipulate data later. 

Tidyverse has many packages that help deal with different types of vectors much more efficiently than the functions that Base R provides. With the stringr package, you can wrangle strings (or characters) and regular expressions. With forcats, you can wrangle factor, the ordered data. With the lubridate package, you can wrangle dates and times.  

Upon successful completion of this module, you will be able to:

1. Explain atomic R data types - numeric, character, logical, factors, and dates.
2. Explain the differences between data structures -- vectors, matrices, data frames, and lists.
3. Understand when to use each data type.
4. Create data frames.
5. Use popular functions from base R and Tidyverse to view and manipulate data frames that contain the various data types.
6. Use popular functions from stringr package to handle strings and regular expressions.
7. Use popular functions from the forcats package to manipulate factors in R.
8. Use popular functions from the lubridate package to manipulate dates.

Building on what you learned in the previous module, you will deepen your understanding of the R data types with particular emphasis on tidyverse.

You will develop a practical sense of choosing appropriate functions from various packages associated with certain data types (stringr, forcats, and lubridate) by watching the live performances and well-prepared demonstrations on using those packages.  Further, you will also be given the opportunity to drill with more advanced topics with those packages. In addition, building on the table creation skills you learned in earlier modules, you will learn how to customize the table extensively to support your imagination and visualization goals, using gt package. Finally, you will learn how to easily produce publication-ready quality statistics summary table and style them with gtsummary package. 

Upon successful completion of this module, you will be able to:

1. Use advanced functions from base R and Tidyverse to view and manipulate data frames that contain the various data types.
2. Use advanced functions from stringr package to handle strings and regular expressions.
3. Use advanced functions from the forcats package to manipulate factors in R.
4. Manipulate dates and times using advanced functions from the lubridate package.
5. Utilize cheat cheats available online to find the right functions quickly.
6. Highly customize great tables with gt package in any way you want.
7. Create presentation-ready statistics summary tables with gtsummary package.

Being able to import different formats of data is important because data scientists deal with secondary data collected by others, and secondary data exist everywhere inside and outside the organizations they work for. Therefore, learning how to import and export data is a fundamental step that we are introducing after you learn about data types and structures in Modules 7 and 8.  You must use appropriate tools to import and wrangle the data depending on the data formats.  Also, gathering data outside your company, such as websites, is a great way to extend your organization's capability, and you will be in the center of the action, adding value to your company; thus, web scraping can be another form of importing data that you can have under your took box.  Further, you will learn how to deal with so-called labelled data produced by menu-driven proprietary statistical software such as SAS, SPSS, and STATA.

In addition to the Base R functions, Tidyverse has some packages that help you deal with different formats of data created by well-known software. 

Upon successful completion of this module, you will be able to:

1. Explain how to create a Github repository and collaborate with others on the same R projects.
2. Effectively load and look through built-in datasets in R.
3. Import various formats of data (csv, xlsx, and SPSS) to RStudio.
4. Recognize basic HTML and CSS terminology. 
5. Scrape data from the web using SelectorGadget and rvest package.
6. Scrape the web data using inspect function of a browser and rvest.
7. Import multiple external data sets and work with them.
8. Work with labeled data in R.
9. Export/save output data to local pc and push to Github.

Continuing on literate coding and reproducible publication paradigm and the notion of strategic data visualization, we will go deeper into data presentation and report sharing in various formats using Quarto and R. That is, you will learn how to build a stunning Quarto dashboard tailored for your audience. In addition, you will learn how to design an interactive web app with Shiny in R. Moreover, you will practice using Github to share your work with the public for reproducibility. You will also learn how to use Github to create your website or blogs and maintain them for free using some basic HTML/CSS as well as htmltools package in R.

Upon successful completion of this module, you will be able to:

1. Explain how Quarto facilitates publication without having extensive knowledge of HTML/CSS. 
2. Create a Quarto dashboard. 
3. Describe how to make an interactive web app with Shiny R. 
4. Be comfortable in using Github to share your work. 
5. Publish a website to Github and style the website to Github from Quarto. 
6. Produce customized tables from imported SPSS data effectively in R. 
7. Create charts from imported SPSS data in R. 

The ability to create a function of your own will make you get even closer to being a data scientist. Although R has several built-in functions of high usage, you will want to be able to create your own functions tailored for your job or tasks, because you found yourself or your team performing a particular task repeatedly. By building your own function, you can help save time. 

In this module, you will be introduced to various types of vectors, logical operators, for-loops, if-else statements. You will also be introduced to good coding practice and see how this positively impacts your code's efficiency. For your code to be more readable, take fewer keystrokes, and execute a batch of jobs faster, you may want to use a family of map() functions from purrr package, which is part of Tidyverse, or use apply() family functions from base R. Furthermore, you will also be introduced to some built-in R functions, which are built by R architecture, using the same sets of principles you will learn in this module. After you finish this module, you should be considered a programmer.

Upon successful completion of this module, you will be able to:

1. Describe good coding practices.
2. Use the various types of logical operators.
3. Use for loops in conjunction with other statements (if-else, next, break, etc).
4. Create your own R functions.
5. Use map functions from the purrr package to increase efficiency.
6. Use a family of apply() functions to simplify repetitive tasks.
7. Be familiar with some built-in R functions.

Building on the fundamentals of programming you learned in the previous module, you will deepen your understanding of R programming in this module. You will also practice more on a family of map() functions from purrr package.  Further, you will be exposed to more packages and tools that allow you to go beyond the familiar Tidyverse ecosystem. 

Specifically, we will go deeper into Functions and iterations with more explanations and examples to practice. Then, you will learn more about the purr package and its family of map functions with demonstrations and exercises, including walk(), walk2(), pwalk().

As the concept is hard, you will be given many examples with which you can appreciate the concepts better. For instance, you will be introduced to a way to test thousands of statistical tests using the purrr and broom packages.  You will also see that you can use purr and reactive packages to make your quarto dashboard interactive. 

Furthermore, you will learn how data scientists can automate Quarto reports with parameters. Finally, you will learn how to solve dashboard fatigue among executives with parameterized reporting brought to the online venue where they spend the most time. 

Upon successful completion of this module, you will be able to:

1. Explain a good coding style.
2. Describe all components of a function and their roles.
3. Set return values and describe the environment in a function.
4. Explain the differences between various For loop variations.
5. Map over multiple arguments using map2() and pmap().
6. Use walk(), walk2(), and pwalk() in the middle of the pipelines.
7. Create an interactive dashboard with Quarto. 
8. Implement parameterized reporting with Quarto to automate business routine operations. 
9. Describe how to avoid dashboard fatigue among decision-makers. 

Working on a hands-on project is a great way to solidify your learning. Such a comprehensive, practical project can be added to a resume, portfolio, or LinkedIn profile. Use all the tools you learned from all 12 modules. Sometimes, you may have to use additional materials given in each module.

For the capstone project, participants will be asked to use the topic and data of their choice to make the project useful for their careers and jobs. Participants will use the regular project workflow they have learned to prepare, analyze, and visualize the data. They will also state the insights from the data in the Quarto document. To realize reproducible reports, students will publish their reports and dashboards on either websites or blogs in GitHub. This can be a souvenir that they can use to showcase their capabilities. 

Upon successful completion of this module, you will be able to:

1. Be familiar with R data structure.
2. Perform various operations to view and manipulate data
3. Import various types of data and export output data to other types of data.
4. Wrangle data to “tidy” form for visualization and modeling.
5. Write codes to automate/simplify routine operation.
6. Create charts, using R’s built-in functions as well as popular packages such as ggplot2.
7. Create tables using gt, gtExtra, and gtsummary packages.  
8. Create a dashboard. 
9. Put together quarto documents (HTML, pdf, Word) and publish them on the web as a website or blog.