MEASUREMENT THEORY
Second Assignment
Due 15 November 2004

1. In this problem we are going to process junk1.txt -- the header for the roll call file -- so that the legislator ID numbers are properly left-justified. To begin, download the header file:
   
   Header.TXT -- Header File for Chile Roll Call Data
   
   save it in a directory, and bring it up in Epsilon. It should look like this:
   
   
   
   We are going to write a simple Keyboard Macro that will insert some spaces as the beginning of line, search forward for "UCSD", then space back to about 4 spaces to the left of the ID number, do a carriage return (Enter or its equivalent, C-J, see Line Translation Page of the Epsilon On-Line Manual for an explanation), go up to the broken line (which will consist of blanks), delete it, and then delete the carriage return. This will left justify the number. Note that inserting the "     ucsd     " string for the TABs (TAB is also C-I) makes all of this possible (see Indenting Commands of the Epsilon On-Line Manual for more details).
   
   To begin the macro, use the start-keyboard-macro command:
   
   C-X (
   
   where "(" is the left parenthesis. Hit the Space Bar 10 times:
   
   
   
   Now, search forward -- C-S ucsd -- for "ucsd" (you do not need to put spaces before and after as we did in Homework 1).
   
   
   
   Now back-up 12 spaces using either the [LEFT] arrow key or C-U12 C-B -- C-B 12 times. (Note that when you back-up 7 spaces the cursor should be adjacent to the "1".) You should be here:
   
   
   
   Now type Enter to break the line:
   
   
   
   Use the [LEFT] arrow key or C-B to back up one space. This puts you at the end of the line of spaces:
   
   
   
   Go to the beginning of the line with the command C-A, "kill" the line of spaces with C-K, type C-D or Delete and you should be here:
   
   
   
   To complete the macro, move down one line with the command C-N and close the macro with the command C-X ). It should look like this:
   
   
   
   Run the macro one time with C-X E and the cursor should be at the beginning of the third line:
   
   
   
   Recall that you can find out how many lines there are in a file with the count-lines command, C-X L. There are 121 lines in the file so if you execute the macro 118 more times using the command C-U118 then C-X E, you should get:
   
   
   
   Run the macro one more time with C-X E and save the file with C-X C-S. Go to the beginning of the file with either ESC < (hit the Escape Key and then type < (less than sign)) or Alt-< (hold Alt Key down and type <). This will put you at the top of the file:
   
   
   
   Note that the ID numbers are left justified and the "ucsd"s to their immediate right are a nice neat column.
   
   E-Mail me your file for a PASS on this part of the homework.
   
   Note that you can use a variation of this macro to line up the member names as well as left-justify the party ID codes.
   
   
2. This problem is a variation on problem 2 of Homework 1. We are going to repeat problem 1 with the keyboard macro described above as two command files -- one command file calls the other! Epsilon is very powerful software that can be used in a wide variety of contexts. With nested macros you can do almost anything to a file. For example, I use them to process the roll call vote webpages from Thomas into text files with the roll call votes as "1"s, "6"s, "7"s (present), and "9"s (not voting).
   
   Download these three macro files
   
   Homework-2-2.TXT -- Keyboard Macro as Text File (Version 1)
   
   Homework-2-2A.TXT -- Keyboard Macro as Text File (Version 2)
   
   Spaces2.TXT -- Keyboard Macro to Insert Spaces
   
   Bring header.txt up in Epsilon, split the window using the C-X 2 command, use the find-file C-X C-F command to bring up Homework-2-2A.TXT, split the window again, and bring up Spaces2.TXT. You should see this:
   
   
   
   Note that spaces is called by homework-2-2A! The spaces macro does one space bar. The command Space is a command within Epsilon that inserts a special character -- in this case, the Space Bar (Note that because of the limitations of HTML I had to omit the "<" and ">" around Space.)
   
   Because homework-2-2A calls Spaces, you must load the spaces2.txt file first as we did in homework 1. With the cursor in the bottom window type Alt-X (Hold Alt Key down and type X) and after the Comand: banner comes up, type load-buffer and hit the Enter key. You should get 0 errors detected.
   
   Move the cursor up to homework-2-2A.txt using C-X P and load this file as well using Alt-X and load-buffer. Finally, move the cursor up to header.txt using C-X P. You should now be here:
   
   
   
   To run the macro type Alt-X and at the Comand: banner type homework-2-2A (the name in the quotes) and hit Enter. You should get:
   
   
   
   Run the macro 120 more times and you will get a file identical to problem 1. Save this file as header2.txt.
   
   Bring header.txt up in Epsilon, split the window using the C-X 2 command, use the find-file C-X C-F command to bring up Homework-2-2.TXT. You should see this:
   
   
   
   Note that this macro is simpler in structure than the pair of macros above. Instead of calling spaces which in turn uses Space to do the Space Bar, I put Space directly in the macro.
   
   Also note that the macro is three lines. If you compare it to homework-2-2A.txt you will see that the line is broken where the two C-M's appear. C-M is equivalent to the Enter Key. Hence, Epsilon "sees" an Enter (or C-M) after the ucsd on the first line, and an Enter (or C-M) after the C-B on the second line.
   
   With the cursor in the bottom window type Alt-X and then load-buffer and hit the Enter key. You should get 0 errors detected.
   
   Move the cursor up to header.txt using C-X P. You should now be here:
   
   
   
   Run the macro for all 121 rows and you will get a file identical to problem 1. Save this file as header3.txt.
   
   E-Mail me header2.txt and header3.txt for a PASS on this part of the homework.
   
   
3. In this problem we are going to analyze the processed Chile roll call matrix with my Optimal Classification (OC) Program. Be sure to read all the explanatory matrial on the Optimal Classification Page first!
   
   Download the Chile Roll Call data:
   
   Chile_Roll_Calls.ord -- Chile Roll Call Data
   
   and the Optimal Classification program and its "Control File":
   
   Optimal Classification (OC) Scaling Program Executable (Compiled for WINTEL Machines) -- PERFL.EXE
   
   Control File -- PERFSTRT.DAT
   
   and put all the files in the same directory.
   
   Bring Chile_Roll_Calls.Ord up in Epsilon, split the window using the C-X 2 command and use the find-file C-X C-F command to bring up Perfstrt.dat. You should be here:
   
   
   
   The first line of Perfstrt.dat is the name of the roll call data file and the second line is a title (or banner) that will be printed out in some of the output files. Change the first line to Chile_Roll_Calls.Ord and change the second line to something that you will recognize (this is important for record-keeping purposes!!). You should be here:
   
   
   
   The next thing you need to do is change the third line of Perfstrt.dat so that it tells OC the correct number of roll calls in Chile_Roll_Calls.Ord, the number of dimensions to estimate, and the format of Chile_Roll_Calls.Ord.
   
   It is easy to determine the number of roll calls (although you could always laboriously count them!!) using the Show-Point Command -- C-X = (hold Control Key down and type X, and then type = ) -- in Epsilon. Specifically, put the cursor in the top window adjacent to the first roll call on line one and type C-X =. You should get this:
   
   
   
   The first number in the red banner in the command line in the bottom of the window -- Column 51, char 51 of 36663 is '1'=49 decimal=31 hex -- tells you the number of columns to the left of the cursor.
   
   Now go to the end of the line with C-E and do C-X = again. You should get this:
   
   
   
   The first number in the red banner in the command line in the bottom of the window -- Column 302, char 302 of 36663 is '^J'=10 decimal=0A hex -- is now 302. In other words, there are 302 characters (columns of width one) in the first line. Hence the number of roll calls must be:
   
   302 - 51 = 251
   
   and note that there are 51 columns before the first roll call in the file.
   
   Go back to Perfstrt.dat and change the number of roll calls to 251 and change the "36" in lines three, four, and five, to "51". You should now be here:
   
   
   
   Finally, change the "10" and the "18" in line three to " 1" and " 1". As I explain on the Optimal Classification Page these two numbers tell OC what legislator to place on the left side of dimension one and what legislator to put on the upper side of dimension two (technically, it simply tells the program to make sure that the coordinates are less than zero and greater than zero, respectively). You should now have:
   
   
   
   Make sure that you save Perfstrt.dat -- C-X C-S or the usual WINDOZE file-save icon -- and exit Epsilon -- C-X C-C.
   
   A word of caution -- as I explain on the Optimal Classification Page the numbers in the third line of Perfstrt.dat are in fixed format. That is, the line:

       2  251   20   51   10   18   10 0.005
   
   is seen by the computer as:
   
   0000200251000200005100010000180001000.005

   So do not change the relative spacing of the numbers when you change 1 (00001) to 10 (00010)! An easy way to do this is to use the Insert key. Within Epsilon if you hit the Insert key it changes the cursor into a flashing rectangle. It looks like this:
   
   
   
   You can now overstrike whatever character the cursor covers (exactly how the Insert key works within MICROSHAFT Word).
   
   At the WINDOZE command line type
   
   PERFL
   
   The program should only take a few seconds (probably a few 1/10's of a second) to run and it produces three output files -- PERF21.DAT, PERF23.DAT, and PERF25.DAT. PERF21.DAT should look something like this:

   
    30 OCTOBER   2004  16.05.38.76.
    RANDOM NUMBER SEED     59100
   CHILE_ROLL_CALLS.ORD                                            
   NON-PARAMETRIC MULTIDIMENSIONAL UNFOLDING OF CHILE DATA         
       1  251   20   51    1    1   10 0.005
   (51A1,3900I1)                                                   
   (I5,1X,51A1,2I5,50F8.3)                                         
    ******************************************************************************
     1 ROLL CALLS   1     764   13793  0.05539  0.94461  0.78295
     2 LEGISLATORS  1     697   13793  0.05053  0.94947  0.80199  0.98774
     3 ROLL CALLS   1     679   13793  0.04923  0.95077  0.80710
     4 LEGISLATORS  1     664   13793  0.04814  0.95186  0.81136  0.99741
     5 ROLL CALLS   1     662   13793  0.04800  0.95200  0.81193
     6 LEGISLATORS  1     658   13793  0.04771  0.95229  0.81307  0.99952
     7 ROLL CALLS   1     655   13793  0.04749  0.95251  0.81392
     8 LEGISLATORS  1     652   13793  0.04727  0.95273  0.81477  0.99783
     9 ROLL CALLS   1     648   13793  0.04698  0.95302  0.81591
    10 LEGISLATORS  1     643   13793  0.04662  0.95338  0.81733  0.99961
    11 ROLL CALLS   1     641   13793  0.04647  0.95353  0.81790
    12 LEGISLATORS  1     640   13793  0.04640  0.95360  0.81818  0.99879
    13 ROLL CALLS   1     639   13793  0.04633  0.95367  0.81847
    14 LEGISLATORS  1     638   13793  0.04626  0.95374  0.81875  0.99893
    15 ROLL CALLS   1     638   13793  0.04626  0.95374  0.81875
    16 LEGISLATORS  1     638   13793  0.04626  0.95374  0.81875  0.99998
                         16.05.38.78.
    ELAPSED TIME OF JOB  16.05.39.53.

   In one dimension you should reproduce the above almost exactly (its possible that some machines might produce slightly different results). Note that you can check your results against the above by scraping this webpage with Epsilon and doing a file comparison using C-F2. Specifically, here is a screen shot of my doing just that:
   
   
   
   Put the highlighted text on the clipboard (right-click in WINDOZE then copy), bring up your copy of PERF21.DAT in Epsilon, split the window using the C-X 2 command and use the find-file -- C-X C-F -- command to bring up an empty file -- scrape.txt -- then paste what you scraped into the file. It should look something like this:
   
   
   
   Go to the beginning of Scrape.txt with Alt-< (hold Alt key down and type < ) or C-Home (hold Ctrl Key down and press the Home key) and you can now do the file comparison:
   
   
   
   Note that the first two lines are clearly different. When you do a file comparison you will get this:
   
   
   
   This is why this command in Epsilon is so handy! Note where the cursor is. If you move to the file in the upper window with C-X [UP] or C-X P the cursor will be in the same position in the other file:
   
   
   
   Position the cursor at the beginning of the third line in both files and then do the file comparison. You should get:
   
   
   
   This is what we want -- the output is the same except for the date, time, and random number stamps.
   
   Save your scrape.txt file and the output files PERF21.DAT, PERF23.DAT, and PERF25.DAT (rename them because they will be overwritten when you run OC again!).
   
   Use Epsilon to change the number of dimensions from "1" to "2" in Perfstrt.dat (the first number in line three) and run OC again.
   
   E-Mail me scrape.txt and PERF21.DAT from the 2-dimensional run for a PASS on this part of the homework.
   
   Save everything as we will be using it to do graphs in R in the next homework!