This is a guide to importing and exporting data to and from R.
This manual is for R, version 3.5.1 Patched (2018-09-11).
Copyright © 2000–2018 R Core Team
Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.
Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.
Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the R Core Team.
|• Spreadsheet-like data:|
|• Importing from other statistical systems:|
|• Relational databases:|
|• Binary files:|
|• Image files:|
|• Network interfaces:|
|• Reading Excel spreadsheets:|
|• Function and variable index:|
|• Concept index:|
The relational databases part of this manual is based in part on an earlier manual by Douglas Bates and Saikat DebRoy. The principal author of this manual was Brian Ripley.
Many volunteers have contributed to the packages used here. The principal authors of the packages mentioned are
DBI David A. James dataframes2xls Guido van Steen foreign Thomas Lumley, Saikat DebRoy, Douglas Bates, Duncan Murdoch and Roger Bivand gdata Gregory R. Warnes ncdf4 David Pierce rJava Simon Urbanek RJDBC Simon Urbanek RMySQL David James and Saikat DebRoy RNetCDF Pavel Michna RODBC Michael Lapsley and Brian Ripley ROracle David A, James RPostgreSQL Sameer Kumar Prayaga and Tomoaki Nishiyama RSPerl Duncan Temple Lang RSPython Duncan Temple Lang RSQLite David A, James SJava John Chambers and Duncan Temple Lang WriteXLS Marc Schwartz XLConnect Mirai Solutions GmbH XML Duncan Temple Lang
Brian Ripley is the author of the support for connections.
Reading data into a statistical system for analysis and exporting the results to some other system for report writing can be frustrating tasks that can take far more time than the statistical analysis itself, even though most readers will find the latter far more appealing.
This manual describes the import and export facilities available either in R itself or via packages which are available from CRAN or elsewhere.
Unless otherwise stated, everything described in this manual is (at least in principle) available on all platforms running R.
In general, statistical systems like R are not particularly well suited to manipulations of large-scale data. Some other systems are better than R at this, and part of the thrust of this manual is to suggest that rather than duplicating functionality in R we can make another system do the work! (For example Therneau & Grambsch (2000) commented that they preferred to do data manipulation in SAS and then use package survival in S for the analysis.) Database manipulation systems are often very suitable for manipulating and extracting data: several packages to interact with DBMSs are discussed here.
There are packages to allow functionality developed in languages such as
python to be directly integrated
with R code, making the use of facilities in these languages even
more appropriate. (See the rJava package from CRAN
and the SJava, RSPerl and RSPython packages from the
Omegahat project, http://www.omegahat.net.)
It is also worth remembering that R like S comes from the Unix
tradition of small re-usable tools, and it can be rewarding to use tools
perl to manipulate data before import or
after export. The case study in Becker, Chambers & Wilks (1988, Chapter
9) is an example of this, where Unix tools were used to check and
manipulate the data before input to S. The traditional Unix tools
are now much more widely available, including for Windows.
This manual was first written in 2000, and the number of scope of R packages has increased a hundredfold since. For specialist data formats it is worth searching to see if a suitable package already exists.
|• Export to text files:|
The easiest form of data to import into R is a simple text file, and
this will often be acceptable for problems of small or medium scale.
The primary function to import from a text file is
scan, and this
underlies most of the more convenient functions discussed in
However, all statistical consultants are familiar with being presented by a client with a memory stick (formerly, a floppy disc or CD-R) of data in some proprietary binary format, for example ‘an Excel spreadsheet’ or ‘an SPSS file’. Often the simplest thing to do is to use the originating application to export the data as a text file (and statistical consultants will have copies of the most common applications on their computers for that purpose). However, this is not always possible, and Importing from other statistical systems discusses what facilities are available to access such files directly from R. For Excel spreadsheets, the available methods are summarized in Reading Excel spreadsheets.
In a few cases, data have been stored in a binary form for compactness and speed of access. One application of this that we have seen several times is imaging data, which is normally stored as a stream of bytes as represented in memory, possibly preceded by a header. Such data formats are discussed in Binary files and Binary connections.
For much larger databases it is common to handle the data using a database management system (DBMS). There is once again the option of using the DBMS to extract a plain file, but for many such DBMSs the extraction operation can be done directly from an R package: See Relational databases. Importing data via network connections is discussed in Network interfaces.
Unless the file to be imported from is entirely in ASCII, it
is usually necessary to know how it was encoded. For text files, a good
way to find out something about its structure is the
command-line tool (for Windows, included in
reports something like
text.Rd: UTF-8 Unicode English text text2.dat: ISO-8859 English text text3.dat: Little-endian UTF-16 Unicode English character data, with CRLF line terminators intro.dat: UTF-8 Unicode text intro.dat: UTF-8 Unicode (with BOM) text
Modern Unix-alike systems, including macOS, are likely to produce
UTF-8 files. Windows may produce what it calls ‘Unicode’ files
UCS-2LE or just possibly
UTF-16LE1). Otherwise most files will be in a
8-bit encoding unless from a Chinese/Japanese/Korean locale (which have
a wide range of encodings in common use). It is not possible to
automatically detect with certainty which 8-bit encoding (although
guesses may be possible and
file may guess as it did in the
example above), so you may simply have to ask the originator for some
clues (e.g. ‘Russian on Windows’).
‘BOMs’ (Byte Order Marks,
https://en.wikipedia.org/wiki/Byte_order_mark) cause problems for
Unicode files. In the Unix world BOMs are rarely used, whereas in the
Windows world they almost always are for UCS-2/UTF-16 files, and often
are for UTF-8 files. The
file utility will not even recognize
UCS-2 files without a BOM, but many other utilities will refuse to read
files with a BOM and the IANA standards for
UTF-16BE prohibit it. We have too often been reduced to
looking at the file with the command-line utility
od or a hex
editor to work out its encoding.
utf8 is not a valid encoding name (
macintosh is the most portable name for what is sometimes
called ‘Mac Roman’ encoding.
Exporting results from R is usually a less contentious task, but there are still a number of pitfalls. There will be a target application in mind, and often a text file will be the most convenient interchange vehicle. (If a binary file is required, see Binary files.)
cat underlies the functions for exporting data. It
file argument, and the
append argument allows a
text file to be written via successive calls to
especially if this is to be done many times, is to open a
connection for writing or appending, and
cat to that connection,
The most common task is to write a matrix or data frame to file as a
rectangular grid of numbers, possibly with row and column labels. This
can be done by the functions
write just writes out a matrix or vector in a specified
number of columns (and transposes a matrix). Function
write.table is more convenient, and writes out a data frame (or
an object that can be coerced to a data frame) with row and column
There are a number of issues that need to be considered in writing out a data frame to a text file.
Most of the conversions of real/complex numbers done by these functions
is to full precision, but those by
write are governed by the
current setting of
options(digits). For more control, use
format on a data frame, possibly column-by-column.
R prefers the header line to have no entry for the row names, so the file looks like
dist climb time Greenmantle 2.5 650 16.083 ...
Some other systems require a (possibly empty) entry for the row names, which
write.table will provide if argument
col.names = NA
is specified. Excel is one such system.
A common field separator to use in the file is a comma, as that is
unlikely to appear in any of the fields in English-speaking countries.
Such files are known as CSV (comma separated values) files, and wrapper
write.csv provides appropriate defaults. In some
locales the comma is used as the decimal point (set this in
dec = ",") and there CSV files use the
semicolon as the field separator: use
write.csv2 for appropriate
defaults. There is an IETF standard for CSV files (which mandates
commas and CRLF line endings, for which use
eol = "\r\n"), RFC4180
(see https://tools.ietf.org/html/rfc4180), but what is more
important in practice is that the file is readable by the application it
is targeted at.
Using a semicolon or tab (
sep = "\t") are probably the safest
By default missing values are output as
NA, but this may be
changed by argument
na. Note that
NaNs are treated as
write.table, but not by
By default strings are quoted (including the row and column names).
quote controls if character and factor variables are
quoted: some programs, for example Mondrian
(https://en.wikipedia.org/wiki/Mondrian_(software)), do not accept
Some care is needed if the strings contain embedded quotes. Three useful forms are
> df <- data.frame(a = I("a \" quote")) > write.table(df) "a" "1" "a \" quote" > write.table(df, qmethod = "double") "a" "1" "a "" quote" > write.table(df, quote = FALSE, sep = ",") a 1,a " quote
The second is the form of escape commonly used by spreadsheets.
Text files do not contain metadata on their encodings, so for
non-ASCII data the file needs to be targetted to the
application intended to read it. All of these functions can write to a
connection which allows an encoding to be specified for the file,
write.table has a
fileEncoding argument to make this
The hard part is to know what file encoding to use. For use on Windows,
it is best to use what Windows calls ‘Unicode’2, that is
"UTF-16LE". Using UTF-8 is a good way
to make portable files that will not easily be confused with any other
encoding, but even macOS applications (where UTF-8 is the system
encoding) may not recognize them, and Windows applications are most
unlikely to. Apparently Excel:mac 2004/8 expected
.csv files in
"macroman" encoding (the encoding used in much earlier versions
of Mac OS).
write.matrix in package MASS provides a
specialized interface for writing matrices, with the option of writing
them in blocks and thereby reducing memory usage.
It is possible to use
sink to divert the standard R output to
a file, and thereby capture the output of (possibly implicit)
options(width) setting may need to be increased.
write.foreign in package foreign uses
write.table to produce a text file and also writes a code file
that will read this text file into another statistical package. There is
currently support for export to
When reading data from text files, it is the responsibility of the user to know and to specify the conventions used to create that file, e.g. the comment character, whether a header line is present, the value separator, the representation for missing values (and so on) described in Export to text files. A markup language which can be used to describe not only content but also the structure of the content can make a file self-describing, so that one need not provide these details to the software reading the data.
The eXtensible Markup Language – more commonly known simply as XML – can be used to provide such structure, not only for standard datasets but also more complex data structures. XML is becoming extremely popular and is emerging as a standard for general data markup and exchange. It is being used by different communities to describe geographical data such as maps, graphical displays, mathematics and so on.
XML provides a way to specify the file’s encoding, e.g.
<?xml version="1.0" encoding="UTF-8"?>
although it does not require it.
The XML package provides general facilities for reading and writing XML documents within R. Package StatDataML on CRAN is one example building on XML. Another interface to the libxml2 C library is provided by package xml2.
yaml is another system for structuring text data, with emphasis on human-readability: it is supported by package yaml.
|• Variations on read.table:|
|• Fixed-width-format files:|
|• Data Interchange Format (DIF):|
|• Using scan directly:|
|• Re-shaping data:|
|• Flat contingency tables:|
In Export to text files we saw a number of variations on the format of a spreadsheet-like text file, in which the data are presented in a rectangular grid, possibly with row and column labels. In this section we consider importing such files into R.
read.table is the most convenient way to read in a
rectangular grid of data. Because of the many possibilities, there are
several other functions that call
read.table but change a group
of default arguments.
read.table is an inefficient way to read in
very large numerical matrices: see
Some of the issues to consider are:
If the file contains non-ASCII character fields, ensure that it is read in the correct encoding. This is mainly an issue for reading Latin-1 files in a UTF-8 locale, which can be done by something like
Note that this will work in any locale which can represent Latin-1 strings, but not many Greek/Russian/Chinese/Japanese … locales.
We recommend that you specify the
header argument explicitly,
Conventionally the header line has entries only for the columns and not
for the row labels, so is one field shorter than the remaining lines.
(If R sees this, it sets
header = TRUE.) If presented with a
file that has a (possibly empty) header field for the row labels, read
it in by something like
read.table("file.dat", header = TRUE, row.names = 1)
Column names can be given explicitly via the
names override the header line (if present).
Normally looking at the file will determine the field separator to be
used, but with white-space separated files there may be a choice between
sep = "" which uses any white space (spaces, tabs or
newlines) as a separator,
sep = " " and
sep = "\t". Note
that the choice of separator affects the input of quoted strings.
If you have a tab-delimited file containing empty fields be sure to use
sep = "\t".
By default character strings can be quoted by either ‘"’ or
‘'’, and in each case all the characters up to a matching quote are
taken as part of the character string. The set of valid quoting
characters (which might be none) is controlled by the
sep = "\n" the default is changed to
If no separator character is specified, quotes can be escaped within quoted strings by immediately preceding them by ‘\’, C-style.
If a separator character is specified, quotes can be escaped within quoted strings by doubling them as is conventional in spreadsheets. For example
'One string isn''t two',"one more"
can be read by
read.table("testfile", sep = ",")
This does not work with the default separator.
By default the file is assumed to contain the character string
to represent missing values, but this can be changed by the argument
na.strings, which is a vector of one or more character
representations of missing values.
Empty fields in numeric columns are also regarded as missing values.
In numeric columns, the values
It is quite common for a file exported from a spreadsheet to have all
trailing empty fields (and their separators) omitted. To read such
fill = TRUE.
If a separator is specified, leading and trailing white space in
character fields is regarded as part of the field. To strip the space,
strip.white = TRUE.
read.table ignores empty lines. This can be changed
blank.lines.skip = FALSE, which will only be useful in
fill = TRUE, perhaps to use blank rows to
indicate missing cases in a regular layout.
Unless you take any special action,
read.table reads all the
columns as character vectors and then tries to select a suitable class
for each variable in the data frame. It tries in turn
complex, moving on if any
entry is not missing and cannot be converted.3
If all of these fail, the variable is converted to a factor.
as.is provide greater control.
as.is = TRUE suppresses conversion of character
vectors to factors (only). Using
colClasses allows the desired
class to be set for each column in the input: it will be faster and use
as.is are specified per
column, not per variable, and so include the column of row names
read.table uses ‘#’ as a comment character,
and if this is encountered (except in quoted strings) the rest of the
line is ignored. Lines containing only white space and a comment are
treated as blank lines.
If it is known that there will be no comments in the data file, it is
safer (and may be faster) to use
comment.char = "".
Many OSes have conventions for using backslash as an escape character in text files, but Windows does not (and uses backslash in path names). It is optional in R whether such conventions are applied to data files.
scan have a logical argument
allowEscapes. This is false by default, and backslashes are then
only interpreted as (under circumstances described above) escaping
quotes. If this set to be true, C-style escapes are interpreted, namely
the control characters
\a, \b, \f, \n, \r, \t, \v and octal and
hexadecimal representations like
other escaped character is treated as itself, including backslash. Note
that Unicode escapes such as
\uxxxx are never interpreted.
This can be specified by the
fileEncoding argument, for example
fileEncoding = "UCS-2LE" # Windows ‘Unicode’ files fileEncoding = "UTF-8"
If you know (correctly) the file’s encoding this will almost always
work. However, we know of one exception, UTF-8 files with a BOM. Some
people claim that UTF-8 files should never have a BOM, but some software
(apparently including Excel:mac) uses them, and many Unix-alike OSes do
not accept them. So faced with a file which
file reports as
intro.dat: UTF-8 Unicode (with BOM) text
it can be read on Windows by
read.table("intro.dat", fileEncoding = "UTF-8")
but on a Unix-alike might need
read.table("intro.dat", fileEncoding = "UTF-8-BOM")
(This would most likely work without specifying an encoding in a UTF-8 locale.)
read.table appropriate for CSV and tab-delimited
files exported from spreadsheets in English-speaking locales. The
read.delim2 are appropriate for
use in those locales where the comma is used for the decimal point and
read.csv2) for spreadsheets which use semicolons to separate
If the options to
read.table are specified incorrectly, the error
message will usually be of the form
Error in scan(file = file, what = what, sep = sep, : line 1 did not have 5 elements
Error in read.table("files.dat", header = TRUE) : more columns than column names
This may give enough information to find the problem, but the auxiliary
count.fields can be useful to investigate further.
Efficiency can be important when reading large data grids. It will help
comment.char = "",
colClasses as one of the
atomic vector types (logical, integer, numeric, complex, character or
perhaps raw) for each column, and to give
nrows, the number of
rows to be read (and a mild over-estimate is better than not specifying
this at all). See the examples in later sections.
Sometimes data files have no field delimiters but have fields in pre-specified columns. This was very common in the days of punched cards, and is still sometimes used to save file space.
read.fwf provides a simple way to read such files,
specifying a vector of field widths. The function reads the file into
memory as whole lines, splits the resulting character strings, writes
out a temporary tab-separated file and then calls
This is adequate for small files, but for anything more complicated we
recommend using the facilities of a language like
pre-process the file.
read.fortran is a similar function for fixed-format files,
using Fortran-style column specifications.
An old format sometimes used for spreadsheet-like data is DIF, or Data Interchange format.
read.DIF provides a simple way to read such files. It takes
arguments similar to
read.table for assigning types to each of the columns.
On Windows, spreadsheet programs often store spreadsheet data copied to
the clipboard in this format;
read.DIF("clipboard") can read it
from there directly. It is slightly more robust than
read.table("clipboard") in handling spreadsheets with empty
scan to read the
file, and then process the results of
scan. They are very
convenient, but sometimes it is better to use
scan has many arguments, most of which we have already
read.table. The most crucial argument is
what, which specifies a list of modes of variables to be read
from the file. If the list is named, the names are used for the
components of the returned list. Modes can be numeric, character or
complex, and are usually specified by an example, e.g.
0i. For example
cat("2 3 5 7", "11 13 17 19", file="ex.dat", sep="\n") scan(file="ex.dat", what=list(x=0, y="", z=0), flush=TRUE)
returns a list with three components and discards the fourth column in the file.
There is a function
readLines which will be more convenient if
all you want is to read whole lines into R for further processing.
One common use of
scan is to read in a large matrix. Suppose
file matrix.dat just contains the numbers for a 200 x 2000
matrix. Then we can use
A <- matrix(scan("matrix.dat", n = 200*2000), 200, 2000, byrow = TRUE)
On one test this took 1 second (under Linux, 3 seconds under Windows on the same machine) whereas
A <- as.matrix(read.table("matrix.dat"))
took 10 seconds (and more memory), and
A <- as.matrix(read.table("matrix.dat", header = FALSE, nrows = 200, comment.char = "", colClasses = "numeric"))
took 7 seconds. The difference is almost entirely due to the overhead
of reading 2000 separate short columns: were they of length 2000,
scan took 9 seconds whereas
read.table took 18 if used
efficiently (in particular, specifying
colClasses) and 125 if
Note that timings can depend on the type read and the data. Consider reading a million distinct integers:
writeLines(as.character((1+1e6):2e6), "ints.dat") xi <- scan("ints.dat", what=integer(0), n=1e6) # 0.77s xn <- scan("ints.dat", what=numeric(0), n=1e6) # 0.93s xc <- scan("ints.dat", what=character(0), n=1e6) # 0.85s xf <- as.factor(xc) # 2.2s DF <- read.table("ints.dat") # 4.5s
and a million examples of a small set of codes:
code <- c("LMH", "SJC", "CHCH", "SPC", "SOM") writeLines(sample(code, 1e6, replace=TRUE), "code.dat") y <- scan("code.dat", what=character(0), n=1e6) # 0.44s yf <- as.factor(y) # 0.21s DF <- read.table("code.dat") # 4.9s DF <- read.table("code.dat", nrows=1e6) # 3.6s
Note that these timings depend heavily on the operating system (the basic reads in Windows take at least as twice as long as these Linux times) and on the precise state of the garbage collector.
Sometimes spreadsheet data is in a compact format that gives the covariates for each subject followed by all the observations on that subject. R’s modelling functions need observations in a single column. Consider the following sample of data from repeated MRI brain measurements
Status Age V1 V2 V3 V4 P 23646 45190 50333 55166 56271 CC 26174 35535 38227 37911 41184 CC 27723 25691 25712 26144 26398 CC 27193 30949 29693 29754 30772 CC 24370 50542 51966 54341 54273 CC 28359 58591 58803 59435 61292 CC 25136 45801 45389 47197 47126
There are two covariates and up to four measurements on each subject. The data were exported from Excel as a file mr.csv.
We can use
stack to help manipulate these data to give a single
zz <- read.csv("mr.csv", strip.white = TRUE) zzz <- cbind(zz[gl(nrow(zz), 1, 4*nrow(zz)), 1:2], stack(zz[, 3:6]))
Status Age values ind X1 P 23646 45190 V1 X2 CC 26174 35535 V1 X3 CC 27723 25691 V1 X4 CC 27193 30949 V1 X5 CC 24370 50542 V1 X6 CC 28359 58591 V1 X7 CC 25136 45801 V1 X11 P 23646 50333 V2 ...
unstack goes in the opposite direction, and may be
useful for exporting data.
Another way to do this is to use the function
> reshape(zz, idvar="id",timevar="var", varying=list(c("V1","V2","V3","V4")),direction="long") Status Age var V1 id 1.1 P 23646 1 45190 1 2.1 CC 26174 1 35535 2 3.1 CC 27723 1 25691 3 4.1 CC 27193 1 30949 4 5.1 CC 24370 1 50542 5 6.1 CC 28359 1 58591 6 7.1 CC 25136 1 45801 7 1.2 P 23646 2 50333 1 2.2 CC 26174 2 38227 2 ...
reshape function has a more complicated syntax than
stack but can be used for data where the ‘long’ form has more
than the one column in this example. With
reshape can also perform the opposite transformation.
Some people prefer the tools in packages reshape, reshape2 and plyr.
Displaying higher-dimensional contingency tables in array form typically
is rather inconvenient. In categorical data analysis, such information
is often represented in the form of bordered two-dimensional arrays with
leading rows and columns specifying the combination of factor levels
corresponding to the cell counts. These rows and columns are typically
“ragged” in the sense that labels are only displayed when they change,
with the obvious convention that rows are read from top to bottom and
columns are read from left to right. In R, such “flat” contingency
tables can be created using
which creates objects of class
"ftable" with an appropriate print
As a simple example, consider the R standard data set
UCBAdmissions which is a 3-dimensional contingency table
resulting from classifying applicants to graduate school at UC Berkeley
for the six largest departments in 1973 classified by admission and sex.
> data(UCBAdmissions) > ftable(UCBAdmissions) Dept A B C D E F Admit Gender Admitted Male 512 353 120 138 53 22 Female 89 17 202 131 94 24 Rejected Male 313 207 205 279 138 351 Female 19 8 391 244 299 317
The printed representation is clearly more useful than displaying the data as a 3-dimensional array.
There is also a function
read.ftable for reading in flat-like
contingency tables from files.
This has additional arguments for dealing with variants on how exactly
the information on row and column variables names and levels is
represented. The help page for
read.ftable has some useful
examples. The flat tables can be converted to standard contingency
tables in array form using
Note that flat tables are characterized by their “ragged” display of
row (and maybe also column) labels. If the full grid of levels of the
row variables is given, one should instead use
read.table to read
in the data, and create the contingency table from this using
In this chapter we consider the problem of reading a binary data file written by another statistical system. This is often best avoided, but may be unavoidable if the originating system is not available.
In all cases the facilities described were written for data files from specific versions of the other system (often in the early 2000s), and have not necessarily been updated for the most recent versions of the other system.
|• EpiInfo Minitab SAS S-PLUS SPSS Stata Systat:|
The recommended package foreign provides import facilities for
files produced by these statistical systems, and for export to Stata. In
some cases these functions may require substantially less memory than
write.foreign (See Export to text files) provides an export mechanism with support currently for
EpiInfo versions 5 and 6 stored data in a self-describing fixed-width
read.epiinfo will read these .REC files into
an R data frame. EpiData also produces data in this format.
read.mtp imports a ‘Minitab Portable Worksheet’. This
returns the components of the worksheet as an R list.
read.xport reads a file in SAS Transport (XPORT) format
and return a list of data frames. If SAS is available on your system,
read.ssd can be used to create and run a SAS script that
saves a SAS permanent dataset (.ssd or .sas7bdat) in
Transport format. It then calls
read.xport to read the resulting
file. (Package Hmisc has a similar function
running SAS.) For those without access to SAS but running on Windows,
the SAS System Viewer (a zero-cost download) can be used to open SAS
datasets and export them to e.g. .csv format.
read.S which can read binary objects produced by S-PLUS
3.x, 4.x or 2000 on (32-bit) Unix or Windows (and can read them on a
different OS). This is able to read many but not all S objects: in
particular it can read vectors, matrices and data frames and lists
data.restore reads S-PLUS data dumps (created by
data.dump) with the same restrictions (except that dumps from the
Alpha platform can also be read). It should be possible to read data
dumps from S-PLUS 5.x and later written with
If you have access to S-PLUS, it is usually more reliable to
the object(s) in S-PLUS and
source the dump file in R. For
S-PLUS 5.x and later you may need to use
and to read in very large objects it may be preferable to use the dump
file as a batch script rather than use the
read.spss can read files created by the ‘save’ and
‘export’ commands in SPSS. It returns a list with one
component for each variable in the saved data set. SPSS
variables with value labels are optionally converted to R factors.
SPSS Data Entry is an application for creating data entry
forms. By default it creates data files with extra formatting
read.spss cannot handle, but it is possible to
export the data in an ordinary SPSS format.
Some third-party applications claim to produce data ‘in SPSS format’ but
with differences in the formats:
read.spss may or may not be able
to handle these.
Stata .dta files are a binary file format. Files from versions 5
up to 12 of Stata can be read and written by functions
write.dta. Stata variables with value labels are optionally
converted to (and from) R factors. For Stata versions 13 and later
see CRAN packages readstata13 and haven.
read.systat reads those Systat
SAVE files that are
rectangular data files (
mtype = 1) written on little-endian
machines (such as from Windows). These have extension .sys
or (more recently) .syd.
Octave is a numerical linear algebra system
(http://www.octave.org), and function
package foreign can read in files in Octave text data format
created using the Octave command
save -ascii, with support for
most of the common types of variables, including the standard atomic
(real and complex scalars, matrices, and N-d arrays, strings,
ranges, and boolean scalars and matrices) and recursive (structs, cells,
and lists) ones.
|• Why use a database?:|
|• Overview of RDBMSs:|
|• R interface packages:|
There are limitations on the types of data that R handles well. Since all data being manipulated by R are resident in memory, and several copies of the data can be created during execution of a function, R is not well suited to extremely large data sets. Data objects that are more than a (few) hundred megabytes in size can cause R to run out of memory, particularly on a 32-bit operating system.
R does not easily support concurrent access to data. That is, if more than one user is accessing, and perhaps updating, the same data, the changes made by one user will not be visible to the others.
R does support persistence of data, in that you can save a data object or an entire worksheet from one session and restore it at the subsequent session, but the format of the stored data is specific to R and not easily manipulated by other systems.
Database management systems (DBMSs) and, in particular, relational DBMSs (RDBMSs) are designed to do all of these things well. Their strengths are
The sort of statistical applications for which DBMS might be used are to extract a 10% sample of the data, to cross-tabulate data to produce a multi-dimensional contingency table, and to extract data group by group from a database for separate analysis.
Increasingly OSes are themselves making use of DBMSs for these reasons, so it is nowadays likely that one will be already installed on your (non-Windows) OS. Akonadi is used by KDE4 to store personal information. Several macOS applications, including Mail and Address Book, use SQLite.
Traditionally there had been large (and expensive) commercial RDBMSs (Informix; Oracle; Sybase; IBM’s DB2; Microsoft SQL Server on Windows) and academic and small-system databases (such as MySQL4, PostgreSQL, Microsoft Access, …), the former marked out by much greater emphasis on data security features. The line is blurring, with MySQL and PostgreSQL having more and more high-end features, and free ‘express’ versions being made available for the commercial DBMSs.
There are other commonly used data sources, including spreadsheets, non-relational databases and even text files (possibly compressed). Open Database Connectivity (