na - SparkR-dropna not giving desired output

I have applied the following code on airquality dataset available in R , which has some missing values. I want to omit the rows which has NAslibrary(SparkR)Sys.setenv('SPARKR_SUBMIT_ARGS'='"--packages" "com.databricks:spark-csv_2.10:1.2.0" "sparkr-shell"')sc <- sparkR.init("local",sparkHome = "/Users/devesh/Downloads/spark-1.5.1-bin-hadoop2.6")sqlContext <- sparkRSQL.init(sc)path<-"/Users/devesh/work/airquality/"aq <- read.df(sqlContext,path,source = "com.databricks.spark.csv", header="true", inferSchema="true")head(dropna(aq,how="a...Read more

zoo - Replace NA only within specific range of time series object

I have the following data stored as zoo object: A B C2017-05-31 NA NA 3.12017-06-30 2.5 2.4 3.22017-07-31 2.5 NA 3.32017-08-31 2.6 NA 3.42017-09-30 2.8 2.5 3.62017-10-31 2.3 NA 3.42017-11-30 2.5 NA 3.22017-12-31 2.7 2.6 2.92018-01-31 2.8 NA 3.02018-02-28 2.6 NA NA2018-03-31 NA NA NAYou can reproduce this zoo object as follows:zoo <- data.frame(A=c(NA, 2.5, 2.5, 2.6, 2.8, 2.3, ...Read more

NA impact in Caret Train

Following the example reported in the link below I have the following error: Using nnet for prediction, am i doing it right? Error in = c(0, 0.0998334166468282, 0.198669330795061, : missing values in objectTo solve this error I use the condition na.action = na.omit#Fit modelmodel <- train(y ~ x1 + x2, te, method='nnet', linout=TRUE, trace = FALSE, #Grid of tuning parameters to try: tuneGrid=expand.grid(.size=c(1,5,10),.decay=c(0,0.001,0.1)), na.action = na.omit) ps <- p...Read more

r - Iteratively fill missing values

I have a data frame with one column completely filled and the other one partially filled. I would like like to fill up the nas in the second column with this formula: col3 = col1*lag(col2)When I'm using mutate, it's iterating once. I prefer using a dplyr function since I will need to apply this function over a larger sample.Here is the data framedd <- tibble::tribble(~col1, ~col2, 1.1127493, 34533.98, 0.9432176, 32573.06, 0.9130100, 29758.64, 0.6593648, 19621.80, 0.9222222, 18095.66, 0.7349398, 13299.22, 0.9836066, 13081.20, 0.851851...Read more

missing data - R - Benford - Dataset with NA strings

I've a dataset of macroeconomic data like GDP, inflation, etc... where Rows=different macroeconomic indicators and columns=yearsSince some values are missing (ex: the GDP of any country in any year), they are charged as "NA".When I perform these operations:#data = read.table("14varnumeros.txt", header = FALSE, sep = "", na.strings = "NA", dec = ".", strip.white = TRUE)benford(data, number.of.digits = 1, sign = "both", discrete=TRUE, round=3)#It gives me this error:Error in extract.digits(data, number.of.digits, sign, second.order, discrete = di...Read more

na - Multiple defintions of node[n+1] Openbugs Prediction

I have a likelihood model and a dataset all working nicely, but when it comes to trying to predict an estimate of the posterior predictive distributionfor a certain output, I have no idea what is wrong. I am predicting scores in relation to a set of covariates.I have specific values for these covariates and would like to calculate the output variable using these. I was told that in my data I had to add n+1 to the N observations, and NA to my output list, as well as add my covariates. I was told not to touch the model. The data loads, the model ...Read more