Michael E. Wieczorek Timur Sabitov 2018 raster digital data Wieczorek, M.E., and Sabitov, T.Y., 2019, 30 year (1981 - 2010) annual average duration of consecutive dry and wet days for the Conterminous United States and District of Columbia: U.S. Geological Survey data release, https://doi.org/10.5066/P9ZWQ48W. https://doi.org/10.5066/P9ZWQ48W This metadata record describes the annual average duration of consecutive wet and dry events during the 30-year period 1981–2010 for the conterminous United States. A wet event is defined as a period when the number of consecutive days with precipitation equals or exceeds 1 millimeter. A dry event is defined as a period when the number of consecutive days with precipitation equals 0 millimeters. The source data was produced and acquired from DAYMET (2018) and is presented here as a 1-kilometer resolution GeoTIFF file. This data set was created by the U.S. Geological Survey's (USGS) Hydrologic Systems Team which is part of the USGS National Water Quality Program (NWQP). The spatial variability of 30 year (1981 - 2010) annual duration of consecutive dry and wet days is an important component with respect to water resources and hydrologic studies. Intensity and frequency of precipitation can affect river flow, evapotranspiration, soil moisture, and groundwater recharge (Huntington, 2006). This effort was undertaken to estimate the 30 year (1981 - 2010) annual average duration of consecutive dry and wet days to support statistical analysis, map display, and model parameterization. 19810101 20101231 ground condition Complete None planned -129.7712 -63.1667 54.8088 21.0821 conterminous United States ISO 19115 Topic Category inlandWaters None consecutive dry days consecutive wet days precipitation conterminous United States NHDPlus version 2.1 DAYMET duration of events USGS Metadata Identifier USGS:5c585e8ce4b0708288ff26cc None Conterminous United States None. Please see 'Distribution Info' for details. None. Users are advised to read the data set's metadata thoroughly to understand its appropriate use and data limitations. Michael E Wieczorek U.S. Geological Survey, Northeast Region Geographer mailing address

5522 Research Park Drive

Baltimore MD 21228 United States 443-498-5550 443-498-5510 mewieczo@usgs.gov Thomas G.Huntington 20060306 publication Huntington, T. G. (2006), Evidence for intensification of the global water cycle: Review and synthesis, J. Hydrol., 319, 83–95, doi:10.1016/j.jhydrol.2005.07.003 https://www.sciencedirect.com/science/article/pii/S0022169405003215?via%3Dihub#bib46 No attribute accuracy was performed. Values were spot checked with the source information to ensure the processed values were within the original data ranges . This metadata record describes data for the 30 year (1981 - 2010) annual average number of consecutive dry days where precipitation totals are 0 millimeters and annual average number of consecutive wet days where precipitation equal or exceeds 1 millimeters for the conterminous United States. P.E. Thorton B.W. Mayer Y. Wei R. Devarakonda R.S. Vose R.B. Cook 20180101 publication https://doi.org/10.3334/ORNLDAAC/1328 Digital and/or Hardcopy 19810101 20101231 publication date DAYMET Data provided daily information on precipitation for the conterminous United States. U.S. Environmental Protection U.S. Geological Survey 2012 tabular digital data http://www.horizon-systems.com/NHDPlus/NHDPlusV2_data.php 100000 Digital and/or Hardcopy 2012 observed NHDPlus Provided the hydrologic extent for the conterminous United States. The first step is to extract daily data from the source netcdf files into an ascii raster format using the ESRI geoprocessing tool, MakeNetCDFFeatureLayer_md . The spatial extent of DAYMET netcdf files is for all of North America. In order to construct an output for the area that NHDPlus covers the conterminous United States, a separate coverage of all NHDPlus version 2 catchments (NHD_CONUS.shp) were dissolved into one polygon and used as a clipping extent. The python code below extracts data for each day from the netcdf file and creates a separate ascii file. This was done for all days, for all the years in the period of record, 1981-2010. #Import required libraries import datetime import time import arcpy import os from arcpy import env from arcpy.sa import * from arcpy import Raster from datetime import timedelta, date ncdfPath = r"D:/NETCDF/DAYMET/MainNETCDF/NCDF4/" #Location where netcdf4 files are stored asciiOut = r'D:/NETCDF/DAYMET/MainNETCDF/ASCII/' #Location where ASCII files will be stored conusShape = r"D:/NETCDF/DAYMET/MainNETCDF/ConusShape/Polygon4.shp" #Location of the conus shape file used for clipping extent userarcLoc = r"C:/Users/tsabitov/Documents/ArcGIS/Default.gdb/raster" #Default location of user arcGIS data #Function to extract data in a format that corresponds to the dimension of the existing day in the NetCDF file def daterange(start_date, end_date): for n in range(int ((end_date - start_date).days)): yield start_date + timedelta(n) #Define the range of years of interest for Y in range (1981,2010): #Range of 1981 to 2010 is from available NetCDF files for these years ncFile=ncdfPath+"daymet_v3_prcp_"+str(Y)+"_na.nc4" #daymet_v3_prcp is the pattern of NetCDF file names, str (Y) is string of variable Y which is current year inside of the loop varPrcp = "prcp" # netCDF variable to be processed xDim = "x" #netCDF coordinate variable used to define the x, or longitude, coordinates of the output layer yDim = "y" #netCDF coordinate variable used to define the y, or latitude, coordinates of the output layer bandDim = "" #Band dimension leave blank selectVal = "BY_VALUE" #Specifies the dimension value selection method, BY_VALUE = The input value is matched with the actual dimension value start_date = date(Y, 1, 1) #Define the range of dates to loop over and extract files where Y is defined above and 1, 1 refers to January 1 end_date = date(Y+1, 1, 1) #The end data of the inner loop is the first day of the next year, therefore Y+1 and 1, 1 refers to January 1 i=1 arcpy.env.overwriteOutput = True #Allow overwriting files, to avoid storage issues with intermediate raster files #Variable newpath is pointing to the output folder that will contain data for that year, for example (1981) newpath = asciiOut+str(Y) #This part will create a folder for example (1981) inside of the folder ASCII if it does not exist if not os.path.exists(newpath): os.makedirs(newpath) #Loop over the range of dates #Convert raster to ASCII file #Swich to the next Julian date for single_date in daterange(start_date, end_date): #Create a string that stores dates yearString = str("time ")+str("'")+str(single_date.strftime("%m/%d/%Y")+ str(" 12:00:00 PM"))+str("'") #Create unique name (for each day) for intermediate rasters rasterName="prcp"+str(i) print (rasterName) #Create raster layer from NetCDF file with defined parameters and store it in user directory of Arc, to avoid issues with permissions rastOutput=arcpy.MakeNetCDFRasterLayer_md(ncFile,varPrcp,xDim,yDim,rasterName,bandDim,yearString,selectVal) #Use NHDPlus shapefile as a clipping extent to the DAYMET data (rectangle dimensions are coming from the dimensions of the shapefile) rastOutputClip =arcpy.Clip_management(in_raster= rastOutput, rectangle="-1950433.44750714 -1899807.32188497 2425566.55249287 1246192.67811504", out_raster=userarcLoc, in_template_dataset=conusShape, nodata_value="-9.999000e+03", clipping_geometry="ClippingGeometry", maintain_clipping_extent="MAINTAIN_EXTENT") arcpy.RasterToASCII_conversion(in_raster=rastOutputClip, out_ascii_file=asciiOut+str(Y)+"/"+str(rasterName)+str(".asc")) i=i+1 20180911 The second step is to estimate the average duration of consecutive dry days. A dry day is defined as any day with no daily precipitation. The consecutive dry days with duration exceeding one day is considered an event. This was done for each day in a year for all years in the period of record, 1981-2010. The R code below calculates the average number of consecutive dry day events which is proportion of number of events over the number of days for each event in a year and finds the average of them for each year. Next, it takes each year and calculates an average duration of consecutive dry days for the period of record and creates a final ascii raster of 30 year (1981 - 2010) annual average duration of consecutive dry days. # Removes all variables loaded into environment rm(list = ls()) # ASCII and Output are folder locations for ascii files and outputs to be stored in ASCII_loc <- "G:/DAYMET/ASCII/" Output_loc <- "G:/DAYMET/Final_Run/Output/" Final_loc <- "G:/DAYMET/Final_Run/Final/" # Loads required packages if they are not available on current machine, otherwise it loads libraries if (!require("pacman")) install.packages("pacman") pacman::p_load( parallel, raster ) # Turn off warning messages options(warn = -1) # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate clustering cl <- makeCluster(no_cores) # In the current loop we are saying that value j follows sequence from 1981 to 2010 by 1 # Ex. 1981,1982,1983 etc... for (j in seq(1981, 2010)) { # Sets working directory into the location of the ASCII files setwd(paste(ASCII_loc, j, sep = "")) # Initialization of the vector variable named xVec as empty (NULL) xVec <- NULL #In order to account for leap years we created if else if condition that adjusts value of the variable (y) if (j == 1984) { y <- 366 } else if (j == 1988) { y <- 366 } else if (j == 1992) { y <- 366 } else if (j == 1996) { y <- 366 } else if (j == 2000) { y <- 366 } else if (j == 2004) { y <- 366 } else if (j == 2008) { y <- 366 } else { y <- 365 } # gc() function is a garbage collection function that empties memory gc() #Define empty vector as NULL xVec <- NULL #For each day, define the folder and file name. # Current loop generates a large matrix with column numbers equal to the number of days # in the year and row numbers equal to the number of cells covering contiguous US # y is the variable that defines number of days in a year for (i in seq(1:y)) { loc <- paste(ASCII_loc, j, sep = "") name <- paste(loc, "/prcp", sep = "") name <- paste(name, i, sep = "") print (i) #Load ascii file as raster x <- raster::raster(paste(name, ".asc", sep = "")) #Convert raster into matrix z <- as.matrix(x) #Convert matrix into vector z <- as.vector(z) #Remove (NA) z <- na.omit(z) #Store output as integer z <- as.integer(z) #Combine empty xVec with output (z) xVec <- cbind(z, xVec) } # Define function to find the average annual duration in days of consecutive dry events that are equal 0mm. dryD <- function(x) { csum <- 0 valList <- NULL j <- 1 for (i in seq (1, length (x))) { if (x[i]==0){ csum <- csum+1 if (i == length(x)){ if (csum<2) {} else {valList[j] <- csum} } } else if (x[i]>0) { if (csum <2) {} else { valList[j] <- csum j <- j+1 } csum <- 0 } } if (is.null (valList)) { return (0) } else { return (mean(valList)) } } # Current function will match cells of calculated values and replace values in input vector of the length as calculated plus NA's rastOverWrite <- function(vecOrig, vecCompute, name) { # Define dimensions of the matrix (number of rows and number of columns) xDim <- dim(vecOrig) # Convert reference input raster into matrix vecOrig <- as.matrix(vecOrig) # Convert reference input matrix into vector format vecOrig <- as.vector(vecOrig) # Replace values that are not na in the vector x(loaded) by values that are in vector y (computed) vecOrig[which(!is.na(vecOrig))] <- vecCompute # Replace all NA with -9999 vecOrig[which(is.na(vecOrig))] <- -9999 # Convert vector into matrix using same dimensions as the original matrix of raster Output <- matrix(round(vecOrig, 1), nrow = xDim[1], ncol = xDim[2]) # Write matrix as ascii file write.table(Output, paste(Output_loc, name, sep = ""), row.names = FALSE, col.names = FALSE) } print ("Running function") # apply in parallels function dryD to each row of the matrix Result_dryD <- parRapply(cl, xVec, dryD) # Read in raster file, serving as reference. x <- raster::raster(paste(name, ".asc", sep = "")) # Store dimensions of the raster file into variable xDim xDim <- dim(x) # Use function rastOverWrite to create asc file of output of the average consecutive duration of precipitation for each year rastOverWrite(x, Result_dryD, paste("avDry", ".asc", sep = paste("_", j, sep = ""))) } # A variable filenames will store list of all files that have names starting with avDry in the folder filenames <- list.files(Output_loc, pattern = "avDry", full.names = TRUE) # apply to the list of the existing files a read.table function and load them into idf ldf <- parLapply(cl, filenames, read.table) # Apply to the list of data frames (ldf) function as matrix to convert tables into matrix format dfMat <- parLapply(cl, ldf, as.matrix) # Apply to the list of the matrix (dfMat) function to convert matrix into a vector format dfVec <- parLapply(cl, dfMat, as.vector) # Combine all vectors from (dfVec) into a large matrix vecTab <- do.call("cbind", dfVec) # Find the mean of each row of the matrix vecMean <- parRapply(cl, vecTab, mean) # Stop cluster of parallel processing stopCluster(cl) # Convert output from vector into matrix with dimensions outputMat <- matrix(round(vecMean, 1), nrow = xDim[1], ncol = xDim[2]) # Save result into working directory write.table(outputMat, file = paste (Final_loc, "/Res_main_dry.asc", sep = ""), row.names = FALSE, col.names = FALSE) # Open connection to the text file fConn <- file(paste (Final_loc, "/Res_main_dry.asc", sep = ""), "r+") # Read in connection and store it as Lines Lines <- readLines(fConn) # Create variable line that stores string of headers of the file line <- c( "ncols 4376", "nrows 3146", "xllcorner -1950433.4475071", "yllcorner -1899807.321885", "cellsize 1000", "NODATA_value -9999" ) # Write list of headers to the top of the txt file writeLines(c(line, Lines, sep = ""), con = fConn) # Close connection close(fConn) closeAllConnections() 20180911 The third step is to estimate the average duration of consecutive wet days. A wet day is defined as any day with precipitation equal or exceeding 1 millimeter. The consecutive wet days with duration exceeding one day, considered as an event. This was done for each day in a year for all years in the period of record, 1981-2010. The R code below calculates the average duration of consecutive wet day events which is proportion of number of events over the number of days for event in a year and finds the average of them for each year. Next, it takes each year and calculates an average duration of consecutive wet days for the period of record and creates a final ascii raster of 30 year (1981 - 2010) annual average duration of consecutive wet days. # Removes all variables loaded into environment rm(list = ls()) # ASCII and Output are folder locations for ascii files and outputs to be stored in ASCII_loc <- "E:/DAYMET/Final_Run/ASCII/" Output_loc <- "E:/DAYMET/Final_Run/Output/" Final_loc <- "E:/DAYMET/Final_Run/Final/" # Loads required packages if they are not available on current machine, otherwise it loads libraries if (!require("pacman")) install.packages("pacman") pacman::p_load( parallel, raster ) # Turn off warning messages options(warn = -1) # In the current loop we are saying that value j follows sequence from 1981 to 2010 by 1 # Ex. 1981,1982,1983 etc... for (j in seq(1981, 2010)) { # Sets working directory into the location of the ASCII files setwd(paste(ASCII_loc, j, sep = "")) # Initialization of the vector variable named xVec as empty (NULL) xVec <- NULL #In order to account for leap years we created if else if condition that adjusts value of the variable (y) if (j == 1984) { y <- 366 } else if (j == 1988) { y <- 366 } else if (j == 1992) { y <- 366 } else if (j == 1996) { y <- 366 } else if (j == 2000) { y <- 366 } else if (j == 2004) { y <- 366 } else if (j == 2008) { y <- 366 } else { y <- 365 } # gc() function is a garbage collection function that empties memory gc() # Calculate the number of cores no_cores <- detectCores() - 1 # Initiate clustering cl <- makeCluster(no_cores) #Define empty vector as NULL xVec <- NULL #For each day, define the folder and file name. # Current loop generates a large matrix with column numbers equal to the number of days # in the year and row numbers equal to the number of cells covering contiguous US # y is the variable that defines number of days in a year for (i in seq(1:y)) { loc <- paste(ASCII_loc, j, sep = "") name <- paste(loc, "/prcp", sep = "") name <- paste(name, i, sep = "") print (i) #Load ascii file as raster x <- raster::raster(paste(name, ".asc", sep = "")) #Convert raster into matrix z <- as.matrix(x) #Convert matrix into vector z <- as.vector(z) #Remove (NA) z <- na.omit(z) #Store output as integer z <- as.integer(z) #Combine empty xVec with output (z) xVec <- cbind(z, xVec) } # Define function to find average duration in days of consecutive wet events that are equal or exceed 1mm. wetD <- function(x) { csum<-0 #Count of days per event valList<-NULL j<-1 for (i in seq (1,length(x))){ if (x[i]>=1) { csum<-csum+1 if (i == length(x)){ if (csum<2) {} else {valList[j] <- csum} } } else if (x[i]==0) { if (csum<2) {} else { valList[j] <- csum j<-j+1 } csum <-0 } } if (is.null(valList)){ return (0) } else { if (length(valList) < 2) {return (valList)} else {return (mean (valList))} } } # Current function will match the cells of two vectors and insert the calculated values of the one vector into the other rastOverWrite <- function(vecOrig, vecCompute, name) { # Define dimensions of the matrix (number of rows and number of columns) xDim <- dim(vecOrig) # Convert reference input raster into matrix vecOrig <- as.matrix(vecOrig) # Convert reference input matrix into vector format vecOrig <- as.vector(vecOrig) # Replace values that are not na in the vector x(loaded) by values that are in vector y (computed) vecOrig[which(!is.na(vecOrig))] <- vecCompute # Replace all NA with -9999 vecOrig[which(is.na(vecOrig))] <- -9999 # Convert vector into matrix using same dimensions as the original matrix of raster Output <- matrix(round(vecOrig, 1), nrow = xDim[1], ncol = xDim[2]) # Write the matrix as ascii file write.table(Output, paste(Output_loc, name, sep = ""), row.names = FALSE, col.names = FALSE) } print ("Running function") # apply in parallels function wetD to each row of the matrix Result_wetD <- parRapply(cl, xVec, wetD) # Read in raster file, serving as reference. x <- raster::raster(paste(name, ".asc", sep = "")) # Store dimensions of the raster file into variable xDim xDim <- dim(x) # Use function rastOverWrite to create asc file of output of the average consequitive duration of precipitation for each year rastOverWrite(x, Result_wetD, paste("avWet", ".asc", sep = paste("_", j, sep = ""))) } # A variable, filenames, will store the list of all files that have names starting with Result_wet in the folder filenames <- list.files(Output_loc, pattern = "avWet", full.names = TRUE) # apply to the list of the existing files, a read.table function and load them into idf ldf <- parLapply(cl, filenames, read.table) # Apply a function to the list of data frames (ldf) function to convert the tables into a matrix format dfMat <- parLapply(cl, ldf, as.matrix) # Apply a function to the list of matrices (dfMat) to convert each matrix into a vector format dfVec <- parLapply(cl, dfMat, as.vector) # Combine all vectors from (dfVec) into a large matrix vecTab <- do.call("cbind", dfVec) # Find the mean of each row of the matrix vecMean <- parRapply(cl, vecTab, mean) # Stop cluster of parallel processing stopCluster(cl) # Convert output from vector into matrix with dimensions outputMat <- matrix(round(vecMean, 1), nrow = xDim[1], ncol = xDim[2]) # Save result into working directory write.table(outputMat, file = paste(Final_loc, "/Result_main_wet.asc", sep = ""), row.names = FALSE, col.names = FALSE) # Open connection to the text file fConn <- file(paste(Final_loc, "/Result_main_wet.asc", sep = ""), "r+") # Read in connection and store it as Lines Lines <- readLines(fConn) # Create a variable (string) that stores the header of the ascii raster file line <- c( "ncols 4376", "nrows 3146", "xllcorner -1950433.4475071", "yllcorner -1899807.321885", "cellsize 1000", "NODATA_value -9999" ) # Write the headers to the top of the txt file writeLines(c(line, Lines, sep = ""), con = fConn) # Close connection close(fConn) closeAllConnections() 20180911 The results from Step 3 are re-projected from WGS 1984 Lambert Conformal Conic to NAD 1983 Albers and then converted from an ASCII raster to GEOTIFF. DAYMET 20181212 Raster Grid Cell 3445 4685 1 Albers Conical Equal Area 29.5 45.5 -96.0 23.0 0.0 0.0 row and column 1000.0 1000.0 meters North_American_Datum_1983 GRS 1980 6378137.0 298.2572221010042 Res_dry.tif Raster geospatial data file. Producer Defined Value 30 year (1981 - 2010) annual average duration of consecutive dry days where precipitation totals are 0 millimeters. Producer Defined 3.5999999046326 102.90000152588 days per event Res_wet.tif Raster geospatial data file. Producer Defined Value 30 year (1981 - 2010) annual average duration of consecutive wet days where precipitation totals are greater than 1 millimeter. Producer Defined 1.5 6.5999999046326 days per event GS ScienceBase U.S. Geological Survey mailing address

Denver Federal Center, Building 810, Mail Stop 302

Denver CO 80225 United States 1-888-275-8747 sciencebase@usgs.gov Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data on any other system or for general or scientific purposes, nor shall the act of distribution constitute any such warranty. Digital Data https://doi.org/10.5066/P9ZWQ48W None 20200821 Michael E Wieczorek U.S. Geological Survey, Northeast Region Geographer mailing address

5522 Research Park Drive

Baltimore MD 21228 United States 443-498-5550 443-498-5510 mewieczo@usgs.gov FGDC Biological Data Profile of the Content Standard for Digital Geospatial Metadata FGDC-STD-001.1-1999