Monday, 12 October 2015

ALTERNATIVE TO IF -THEN ELSE

Most programmers will have used the IF THEN ELSE statements when working with conditional processing. Assigning one value for when the condition is true, and another value for when the condition is false. In these situations you can reduce lines of code by instead using the functions IFC and IFN.

These functions can both create a new variable with assigned values based on whether a condition is true or false. IFC will create a character variable and IFN will create a numeric variable. 


Below are standard templates for using IF THEN ELSE to create numeric/character variables, along with their counterpart in IFN and IFC:

*Standard IF THEN ELSE code (numeric);
if condition then variable=value1;
else variable=value2;
 *Equivalent IFN code;
variable=ifn(condition,value1,value2);
 *Standard IF THEN ELSE code (character);
if condition thenvariable='value1';
else variable='value2';
 *Equivalent IFC code;
variable=ifc(condition, 'value1' ,'value2') ;

You can see that IFN and IFC effectively cut down and rearrange the keywords from IF THEN ELSE code and change it into just one compact line of code.

For example both the following sets of code can be used to create the same variable:

if sex='Male' then sexcd=1;
else sexcd=0;


sexcd=ifn(sex='Male',1,0);

table_1


Either code can be used here and will produce the same result. In order to compare processing times I ran equivalent code five times for each method and recorded the average times. These were run within server SAS version 9.1.3 and showed very little difference in average processing times. However the IFN function has the advantage of requiring less code.

These functions are ideal when you wish to create variables with 2 possible values.

Read More »

Friday, 9 October 2015

Tips and tricks about PROC SQL

INTRODUCTION

PROC SQL is the implementation of the SQL syntax in SAS. It first appeared in SAS 6.0, and since then has been very popular for SAS users. SAS ships with a few sample data sets in its HELP library, and SASHELP.CLASS is one of them. This dataset contains 5 variables including name, weight, height, sex and age for 19 simulated teenagers, and in this paper I primarily use it for the demonstration purpose. Here I summarize the 10 interesting tricks and tips using PROC SQL. At the beginning, I first make a copy of SASHELP.CLASS at the WORK library and transform the row number of the data set to a new variable obs.
data class;
   set sashelp.class;
   /* Give an index for each child*/
   obs = _n_;
run;

1. Calculate the median of a variable

With the aggregating HAVING clause and some self-join techniques, PROC SQL can easily calculate the median for a variable.

proc sql;
   select avg(weight) as Median
   from (select e.weight
   from class e, class d
   group by e.weight
   having sum(case when e.weight = d.weight then 1 else 0 end)
      >= abs(sum(sign(e.weight - d.weight))));
quit;

2. Draw a horizontal histogram
A histogram visualizes the distribution pattern of a variable. PROC SQL can draw a horizontal histogram by showing the frequency bars with a few asterisks for each level of the variable age.

proc sql;
   select age, repeat('*',count(*)*4as Frequency
   from class
   group by age
   order by age;
quit;

3. Return the running total for a variable
A running total is the summation of a sequence of numbers which is updated each time with the increase of the observations. In the example below, I calculate the running total and save them as a new variable Running_total by the SUM function and a conditional statement, which logically is similar to an example in SAS/IML[1]. 

proc sql;
   select name, weight,
      (select sum(a.weight) from class as
      a where a.obs <= b.obs) as Running_total
   from class as b;
quit;

4. Report the total number for a variable
PROC SQL is a flexible way to find the total number for any variable by its set operator UNION and the SUM function. In the example, the total number of the variable weight is reported at the bottom of the output table.

proc sql;
   select name, weight
   from class
   union all
   select 'Total', sum(weight)
   from class;
quit;
5. Retrieve the metadata for a data set
SAS stores the metadata at its DICTIONARY data sets. PROC SQL can visit the directory, retrieve the column detail, and return the information to the users.
proc sql;
   select name, type, varnum
   from sashelp.vcolumn
   where libname = 'WORK' and memname = 'CLASS';
quit;
6. Rank a variable 
Besides the designated ranking procedure PROC RANK in SAS, PROC SQL can also do some simple ranking as well.

proc sql;
   select name, a.weight, (select count(distinct b.weight)
   from class b
   /* Rank by the ascending order for the weight variable*/
   where b.weight <= a.weight) as rank
   from class a;
quit;
7. Simple random sampling 
PROC SQL is widely used in simple random sampling. For example, I randomly choose 8 observations by the OUTOBS option at the PROC statement. The randomization process is realized by the RANUNI function at the ORDER BY statement with a seed 1234.

proc sql outobs = 8;
   select *
   from class
   order by ranuni(1234);
quit;
8. Replicate a data set without data
In PROC SQL, it is a fairly straightforward one-line statement to create a new empty data set while keeps all the structure of the original data set.
proc sql;
   create table class2 like class;
quit;

9. Transpose data
Usually DATA step ARRAY and PROC TRANSPOSE allow SAS users to restructure the data set, while PROC SQL sometimes is an alternative solution. For instance, if we need a wide-to-long operation to list the names of the children by their gender in the CLASS date set, then PROC SQL can fulfill the functionality through the combinations of some queries and subqueries.

proc sql;
   select max(case when sex='F'
      then name else ' ' endas Female,
      max(case when sex='M'
      then name else ' ' endas Male
   from (select e.sex,
      e.name,
      (select count(*) from class d
      where e.sex=d.sex and e.obs < d.obs) as level
      from class e)
   group by level;
quit;
10. Count the missing values
Another advantage of PROC SQL is that its NMISS function works for both numeric and character variables [2], which makes PROC SQL an ideal tool for missing value detection.

proc sql;
   select count(*) 'Total', nmiss(weight)
      'Number of missing values for weight'
   from class;
quit;
CONCLUSION
The combination of SAS’s powerful functions and the SQL procedure will benefit SAS users in data management and descriptive statistics.
Read More »

Tuesday, 6 October 2015

How to eliminate data error notes from the SAS log

Here's a simple datastep.  Notice the missing dollar sign to indicate the variable GENDER (M, F) is a character variable.
data class;
 infile 'c:\temp\class.csv' dsd;
 input name $ gender age;
run;
We've all seen those ugly data error notes in the SAS log!
562  data class;
563     infile 'c:\temp\class.csv' dsd;
564     input name $ gender age;
565  run;
 
NOTE: The infile 'c:\temp\class.csv' is:
      Filename=c:\temp\class.csv,
      RECFM=V,LRECL=32767,File Size (bytes)=236,
      Last Modified=03Dec2014:12:46:20,
      Create Time=03Dec2014:12:46:20
 
NOTE: Invalid data for gender in line 1 8-8.
RULE:     ----+----1----+----2----+----3----+----4----+----5         
1         Alfred,M,14 11
name=Alfred gender=. age=14 _ERROR_=1 _N_=1
NOTE: Invalid data for gender in line 2 7-7.
2         Alice,F,13 10
name=Alice gender=. age=13 _ERROR_=1 _N_=2
NOTE: Invalid data for gender in line 3 9-9.
3         Barbara,F,13 12
name=Barbara gender=. age=13 _ERROR_=1 _N_=3
NOTE: Invalid data for gender in line 4 7-7.
4         Carol,F,14 10
name=Carol gender=. age=14 _ERROR_=1 _N_=4
NOTE: Invalid data for gender in line 5 7-7.
5         Henry,M,14 10
name=Henry gender=. age=14 _ERROR_=1 _N_=5
NOTE: Invalid data for gender in line 6 7-7.
6         James,M,12 10
name=James gender=. age=12 _ERROR_=1 _N_=6
NOTE: Invalid data for gender in line 7 6-6.
7         Jane,F,12 9
name=Jane gender=. age=12 _ERROR_=1 _N_=7
...
NOTE: 19 records were read from the infile 'c:\temp\class.csv'.
      The minimum record length was 9.
      The maximum record length was 12.
NOTE: The data set WORK.CLASS has 19 observations and 3 variables.
NOTE: DATA statement used (Total process time):
      real time           0.04 seconds
      cpu time            0.04 seconds
Suppressing the data error notes in the SAS log is easy!
options errors=0;
But the above approach only masks the bad news.  That is why the student wanted to write these notes to a separate file.  Here's how!
data class;
 infile 'c:\temp\class.csv' dsd;
 input name $ gender age;
 if _error_=1 then do;
  file 'c:\temp\MyInvalidDataNotes.txt';
  put 'NOTE: Invalid data in line ' _N_;
  put _infile_;
  put _all_;
  put;
 end;
run;
In SAS, there's always a way!
Read More »