May the Force of R be With You, Always!

By Daniel's Blog

center

(This article was first published on Daniel’s Blog, and kindly contributed to R-bloggers)

With my Telegram account connected to @TeleR, the force of R can always be with me, where I have data. The following is a screenshot of my mobile:

If you want to have R where you are too, you will only need a Telegram account, then search for the public username @Tele_R, starting a new chat with. Thanks to TeleMath Team, you will be able to send R commands as messages, and get answers back.

To leave a comment for the author, please follow the link and comment on their blog: Daniel’s Blog.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more…

Source:: R News

Shiny Chart Builder – Explore your database with a point-and-click interface

By Pablo Seibelt

Scatterplot

I have a new year’s surprise for you! This shiny app means to be a system for basic reporting in the style of most Business Intelligence tools, you can create a report without knowing any SQL or R.

This package heavily relies on dplyr for database abstraction, it theoretically works with any dplyr-compatible database, but may require some tuning for some of the databases.

The flow

The way you should use this app is to build your chart with the Sample mode, and when you have the visualization you want, you untick the sample mode, which goes to the database to fetch the complete dataset you need. The app does some tricks with dplyr to avoid over-querying data.

Scatterplot example:

Line chart example:
Linechart

Configuration

The app has to be configured by placing 2 files on the root of the project: config.R and update_samples.R. Example files using sqlite have been provided in the examples folder.

Before using the shiny app, you have to execute the script “update_samples.R”, which will download samples of all tables (Might take a while on big databases). If you only want to query a subset of your tables, modify this script so it only finds those tables.

This script should be reran ocasionally, depending on how much your database changes, maybe daily, weekly or monthly, use your job scheduler or cron to do execute this script.

Also, the stack overflows easily because of the level of recursion used, on the server or machine where you deploy this, you should allow for big stack sizes, i’ve tried and it worked fine with the unlimited setting in my experience. (This command works on Linux, you should find your equivalent in other operating systems if you find stack overflow errors)

ulimit -s unlimited

Missing features

This is a very preliminar release, a lot of things may be missing, pull requests are welcome!

Some examples of missing features:

  • Some advanced settings to control appearance have to be added
  • Bar charts
  • Histograms
  • Faceting maybe?
  • More database examples
  • Bookmarking charts (It fails with the default bookmarker)

Download

The repository you can download this application from is:
https://github.com/sicarul/shiny-chart-builder

Source:: R News

3PL models viewed through the lens of total probability theorem

By Andrés Gutiérrez

Screen Shot 2017 01 01 at 12 46 51 PM

(This article was first published on Data Literacy – The blog of Andrés Gutiérrez, and kindly contributed to R-bloggers)

As I currently am the NPM for PISA in Colombia, I must assist to several meetings dealing with the proper implementation of this assessment in my country. Few of them are devoted to the analysis of this kind of data (coming from IRT models). As usual, OECD has hired organizations with high technical standards. The institute that handles all this data and take part in the analysis is ETS (Educational Testing Services).

PISA and ETS are changing from Rasch models to 2PL models. That involves a significant technical effort to maintain comparability along time. I had the opportunity to talk with some experts from ETS last year, and I formulated them the following question: ¿why to consider 2PL models instead of 3PL models? Well, the answer was not easy, and I am not pretending to explain it here in detail, in part because I am still convinced about the advantages of 3PL models that exceed those of 2PL models. However, they yielded me to a recent paper entitled Is There Need for the 3PL Model? Guess What?

The article was written by Mathias von Davier from ETS. I liked the way von Davier showed 3PL models, as coming from a total probability setup involving 2PL models. Consider the following hierarchical structure: first, the test-taker decides whether he/she is answering that item by guessing or not; then, heshe uses hisher ability to found the correct response. So, the stochastic process behind this structure can be easily shown in a tree diagram:

Remember that 3PL models can be written as:

$$P_{3PL}(x=1) = P(Guess) + P(NoGuess) times P_{2PL}(x=1|NoGuess)$$

Note that, per the model, once the student has chosen to answer by guessing, a correct answer is always found (kind of weird, isn’t it?). So, a major criticism against 3PL models is related to this last point. In R, you can estimate 3PL models by using the mirt package. So, for example, when using the LSAT7 data on the second item, we can estimate this guessing parameter.

library(mirt)
data <- expand.table(LSAT7)
md2 <- mirt(data, 1, itemtype = '3PL', IRTpars = TRUE)
coef(md2, IRTpars = TRUE)$Item.2

We found that the guessing parameter is estimated as 0.295. This way, the model is specified as:

$$P_{3PL}(x=1) = 0.295 + 0.705 times P_{2PL}(x=1|NoGuess)$$

To leave a comment for the author, please follow the link and comment on their blog: Data Literacy – The blog of Andrés Gutiérrez.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more…

Source:: R News

Building Shiny App exercises part 4

By Euthymios Kasvikis

APPLICATION LAYOUT & REACTIVITY
The fourth part of our series is “separated” into two “sub-parts”. In the first one we will start building the skeleton of our application by using tabsetPanel. This is how we will separate the sections of our app and also organize its structure better.
In the second part you will learn hot to load your dataset in RStudio and finally in the third one we will give life to your Shiny App! Specifically, you are going to have your first contact with reactivity and learn how to build reactive output to display in your Shiny app in a form of a data table initially.
Follow the examples below to understand the logic of the tools you are going to use and then enhance the app you started creating in part 1 by practising with the exercise set we prepared for you. Lets begin!

Answers to the exercises are available here.

If you obtained a different (correct) answer than those listed on the solutions page, please feel free to post your answer as a comment on that page.

TABSET PANEL
In the example below you will see hot to add a tabsetPanel in your shiny app.

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs"))
))

#server.R
library(shiny)
shinyServer(function(input, output) {
})

Exercise 1

Add a tabsetPanel to the mainPanel of your Shiny App.

TAB PANEL
In the example below you will see how to add tabPanel in your tabsetPanel.

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1")
))))

#server.R
library(shiny)
shinyServer(function(input, output) {
})

Exercise 2

Place a tabPanel in the tabsetPanel you just added to your Shiny App. Name it “Data Table”.

Exercise 3

Put a second tabPanel next to the first one. Name it “Summary”.

LOAD DATASET
Now it is time to give your app a purpose of existence. This can happen with only one way. To add some data into it! As we told in part 1 we will create an application based on the famous (Fisher’s or Anderson’s) iris data set which gives the measurements in centimeters of the variables sepal length and width and petal length and width, respectively, for 50 flowers from each of 3 species of iris. The species are Iris setosa, versicolor, and virginica.
This is a “built in” dataset of RStudio that will help us create our first application.
But normally you want to analyze your own datasets. The first thing you should do in order to perform any kind of analysis on them is to load them properly. We will now see how to load a dataset in txt form from a local file using RStudio. Let’s go!
The process is quite simple. First of all you have to place the txt file that contains your dataset into the same directory that you are working, secondly press the “Import Dataset” button in RStudio and then “From Local File…”. Find the txt file in your computer and click “Open”, then press “Import”. That’s all! Your dataset is properly loaded in your directory and now you can work with it.
Please note that the purpose of this series is not to teach how to form your dataset before you load it nor how to “clean” it. You can gain more information about this subject from here. Our purpose is to teach you build your first Shiny App.

Exercise 4

Load the dataset you want to analyze (“something.txt”) from your computer to your directory with RStudio buttons.

INTRODUCTION TO REACTIVITY
From this point you are going to enter in the “Kingdom of Reactivity”. Reactive output automatically responds when your user interacts with a widget. You can create reactive output by following two steps. Firstly, add an R object to your ui.R. and then tell Shiny how to build the object in server.R.

1): Add an R object to the UI

Shiny provides a variety of functions that transform R objects into output for your UI as you can see below:

htmlOutput: raw HTML
imageOutput: image
plotOutput: plot
tableOutput: table
textOutput: text
uiOutput: raw HTML
verbatimTextOutput: text

To add output to the UI place the output function inside sidebarPanel or mainPanel in the ui.R script.
For example, the ui.R file below uses tableOutput to add a reactive line of text to “Tab Panel 1”. Nothing happens…for the moment!

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
})

Notice that datatableOutput takes an argument, the character string “dt1”. Each of the *Output functions require a character string that Shiny will use as the name of your reactive element. Users cannot see it and you will understand its role later.

Exercise 5

Add a datatableOutput to “Data Table”, name its argument “Table”.

2): Provide R code to build the object.

The code should be placed in the function that appears inside shinyServer in your server.R script.
This function is of great importance as it builds a list-like object named output that contains all of the code needed to update the R objects in your app. Be careful, each R object MUST have its own entry in the list.
You can create an entry by defining a new element for output within the function. The element name should match the name of the reactive element that you created in ui.R.
Each entry should contain the output of one of Shiny’s render* functions. Each render* function corresponds to a specific type of reactive object. You can find them below:

renderImage: images (saved as a link to a source file)
renderPlot: plots
renderPrint: any printed output
renderTable: data frame, matrix, other table like structures
renderText: character strings
renderUI: a Shiny tag object or HTML

Each render* function takes a single argument: an R expression which can either be one simple line of text, or it can involve many lines of code.
In the example below “dt1” is attached to the output expression in server.R and gives us the Data Table of “iris” dataset inside “Tab Panel 1”.

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
output$dt1 <- renderDataTable(
iris)

})

Exercise 6

Add the appropriate render* function to server.R in order to create the Data table of the “iris” dataset. Hint: Use the output expression.

The dataTableOutput is your first contact with reactivity, in the next parts of our series you will use the rest of the Output functions that Shiny provides. But for now let’s experiment a little bit on this.
As you can see there is a text filter in your Data Table. You can deactivate it by setting searching to be “FALSE” as the example below.

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
output$dt1 <- renderDataTable(
iris,options = list(searching=FALSE))
})

Exercise 7

Disable the Text Filter of your Data Table. Hint: Use options, list and searching.

With the same logic you can disable the pagination that is displayed in your Data Table, as in the example below.

# ui.R

library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
output$dt1 <- renderDataTable(
iris,options = list(searching=FALSE,paging=FALSE))
})

Exercise 8

Disable the Pagination of your Data Table. Hint: Use options, list, paging.
Now you can see how to display an exact number of rows (15) and enable filtering again.

# ui.R

library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
output$dt1 <- renderDataTable(
iris,options = list(pageLength=15))
})

Exercise 9

Enable filtering again and set the exact number of rows that are displayed to 10. Hint: Use options, list, pageLength

We can also create a Length Menu in order to control totally the choices of the numbers of rows we want to be displayed. In the example below we assign every number to a menu label. 5 -> ‘5′, 10 -> ’10’, 15 -> ’15’,-1 -> ‘ALL’.

# ui.R
library(shiny)
fluidPage(
titlePanel("TabPanel"),
sidebarLayout(
sidebarPanel(h3("Menu")),
mainPanel(h3("Main Panel"),tabsetPanel(type = "tabs",
tabPanel("Tab Panel 1",dataTableOutput("dt1")),
tabPanel("Tab Panel 2"))
)))

#server.R
library(shiny)
shinyServer(function(input, output) {
output$dt1 <- renderDataTable(
iris,options = list(
lengthMenu = list(c(5, 15, 25,-1),c('5','15','25','ALL')),
pageLength = 15))
})

Exercise 10

Create a Length Menu with values (10,20,30,-1) and assign each one ot the values to the appropriate menu label. Hint: Use options, list, lengthMenu,pageLength.

Source:: R News

Animations in R using Plotly

By Riddhiman

happy-2017

Like last year, lets have some fun with the Plotly package. We’ll try out Plotly’s new animation capabilities.

library(plotly)
rm(list = ls())
gc()

# Options for plotting ----
x <- 0.2
y <- 0.72
speed <- 250
nbkdrops <- 100

# Colorset for plot
# See http://colorhunt.co/
cols <- c("#FFC85B", "#379956","#234C63")
ncolors <- length(cols)


# Function to create random points by adding jitter to ----
# a starting set of points
n <- 1000  # Number of points

# Starting template
bkdrop.x <- runif(n, min = 0, max = 1)
bkdrop.y <- runif(n, min = 0, max = 1)

# Function Definition
bkdrop <- function(n = 1000, amount = 0.005){
  
  x <- jitter(bkdrop.x, amount = amount)
  y <- jitter(bkdrop.y, amount = amount)
  
  df <- data.frame(x, y)
  
  return(df)
  
}

# Make backdrops ----
# Each call to the backdrop function is a separate frame
# Number of frames is controlled by nbkdrops

bkdrop.df <- data.frame()
for(i in 1:nbkdrops){
  temp <- bkdrop()
  temp <- data.frame(temp, frame = i, color = sample(1:ncolors, size = nrow(temp), replace = T))
  bkdrop.df <- rbind(bkdrop.df, temp)
  
}

# Make back lights ----
# Coordinates for backlight rectangles
# Will be plotted as line segments
bklight.x <- c(0.28, 0.18, 0.48)
bklight.y <- c(0.42, 0.62, 0.65)
bklight.xend <- c(0.63, 0.50, 0.75)
bklight.yend <- c(0.42, 0.62, 0.65)

# Function to create a dataframe containing coordinates, frame and
# color of each backlight segment
makebklight <- function(id){
  bklight <- data.frame()
  
  for(i in 1:nbkdrops){
    temp <- data.frame(x = bklight.x[id],
                       y = bklight.y[id],
                       xend = bklight.xend[id],
                       yend = bklight.yend[id],
                       frame = i, 
                       color = sample(1:ncolors, size = 1))
    
    bklight <- rbind(bklight, temp)
  }
  
  return(bklight)
}

# Create backlight segments
bklight1 <- makebklight(1)
bklight2 <- makebklight(2)
bklight3 <- makebklight(3)

# Initialize colors for first frame
bklight1$color[1] <- 1
bklight2$color[1] <- 2
bklight3$color[1] <- 3

# Plot !! ----
p <- plot_ly(height = 800, width = 1024, 
             colors = cols, 
             frame = ~frame,
             x = ~x, 
             y = ~y,
             color = ~factor(color)) %>%  
  
  # Backdrop
  add_markers(data = bkdrop.df, 
              opacity = 0.8,
              marker = list(symbol = "star", size = 8),
              hoverinfo = "none") %>%
  
  # Add segments (for back lighting)
  add_segments(data = bklight1, 
               xend = ~xend, yend = ~yend, 
               line = list(width = 150)) %>%
  
  add_segments(data = bklight2, 
               xend = ~xend, yend = ~yend, 
               line = list(width = 150)) %>% 
  
  add_segments(data = bklight3, 
               xend = ~xend, yend = ~yend, 
               line = list(width = 150)) %>% 
  
  # Animation options
  # See https://cpsievert.github.io/plotly_book/key-frame-animations.html
  
  animation_opts(speed, easing = "linear", transition = 0) %>%
  animation_button(x = 1, xanchor = "right", y = 1, yanchor = "bottom") %>%
  animation_slider(hide = T) %>%
  
  # Layout, annotations and shapes
  
  layout(
    showlegend = F,
    
    xaxis = list(title = "", showgrid = F, zeroline = F, showticklabels = F, range = c(0, 1)),
    yaxis = list(title = "", showgrid = F, zeroline = F, showticklabels = F, range = c(0, 1)),
    
    annotations = list(
      
      # For shadow
      list(xref = "paper", yref = "paper",
           xanchor = "left", yanchor = "top",
           x = x + 0.002, y = y + 0.002, 
           showarrow = F,
           text = "Happy New<br>Year !",
           font = list(size = 100, family = "Times New Roman",
                       color = "black")),
      
      list(xref = "paper", yref = "paper",
           xanchor = "left", yanchor = "top",
           x = x + 0.003, y = y + 0.003, 
           showarrow = F,
           text = "Happy New<br>Year !",
           font = list(size = 100, family = "Times New Roman",
                       color = "black")),
      
      list(xref = "paper", yref = "paper",
           xanchor = "left", yanchor = "top",
           x = x + 0.004, y = y + 0.004, 
           showarrow = F,
           text = "Happy New<br>Year !",
           font = list(size = 100, family = "Times New Roman",
                       color = "black")),
      
      # Actual
      list(xref = "paper", yref = "paper",
           xanchor = "left", yanchor = "top",
           x = x, y = y, 
           showarrow = F,
           text = "Happy New<br>Year !",
           font = list(size = 100, family = "Times New Roman",
                       color = "#ff6666"))
      ),
    
    shapes = list(
      
      # Border
      list(xref = "paper", yref = "paper",
           x0 = 0, y0 = 0, 
           x1 = 1, y1 = 1,
           type = "rect",
           line = list(width = 10, color = cols[1])),
      
      list(xref = "paper", yref = "paper",
           x0 = 0.01, y0 = 0.01, 
           x1 = 0.99, y1 = 0.99,
           type = "rect",
           line = list(width = 10, color = cols[2])),
      
      list(xref = "paper", yref = "paper",
           x0 = 0.02, y0 = 0.02, 
           x1 = 0.98, y1 = 0.98,
           type = "rect",
           line = list(width = 10, color = cols[3])),
      
      # Black outline
      list(xref = "plot", yref = "plot",
           path = "
           M 0.50 0.53
           L 0.50 0.50
           L 0.18 0.50 
           L 0.18 0.73
           L 0.48, 0.73",
           type = "path",
           line = list(width = 7, color = "black")),
      
      list(xref = "plot", yref = "plot",
           path = "
           M 0.50 0.535
           L 0.48 0.535
           L 0.48 0.77
           L 0.75 0.77
           L 0.75 0.535
           Z",
           type = "path",
           line = list(width = 7, color = "black")),
      
      list(xref = "plot", yref = "plot",
           path = "
           M 0.28 0.5
           L 0.28 0.31
           L 0.63 0.31
           L 0.63 0.535",
           type = "path",
           line = list(width = 7, color = "black"))

    )
  )

print(p)

You should now have something like this:

For mode details visit:
Plotly for R by Carson Seivert.

Source:: R News