Assignment 1. Files and shell scripting

[course home > assignments]

Overview

This assignment is designed to give you familiarity with two things. The first is scripting with POSIX-compatible shells. The second is Emacs, the classic programmable text editor that is a prototype for modern integrated development environments (IDEs). Software developers should be expert in both shell scripting and IDEs, even if they don’t necessarily use POSIXish shells or Emacs per se.

As this course has no textbook, a main goal of this assignment is covering how you can discover details about this assignment’s topic, details that may not be covered in lecture. You can get many of the details by following all the links in this assignment and getting the gist of those web pages (which is something you should do, unless the text around a link says that you don’t need to read all of the referenced document). However, this won’t suffice for everything and you’ll need to do some learning-by-doing to do the assignment well. The idea is that you can put this experience to good use later in this course (and in real life) when you need to come up to speed with a large software ecosystem. That being said, don’t let yourself get discouraged if a detail cannot be found by reading the online documentation. If you need a hint, ask a TA or an LA. (This assignment is not intended to be done without any hints.)

Setup

If you haven’t already, please sign up for a SEASnet account. Do not delay in doing this; new SEASnet accounts may take multiple days to activate. You may already have a SEASnet account if you’re a student in the School of Engineering.

Do this assignment on any of the following SEASnet GNU/Linux servers: lnxsrv11, lnxsrv12, lnxsrv13, or lnxsrv15, with /usr/local/cs/bin prepended to your PATH. You can do this by executing the shell command "export PATH=/usr/local/cs/bin:$PATH" after logging in, or more conveniently by running the helper script mentioned below, which puts that shell command into your $HOME/.profile file (but test this file by logging in via a separate session before logging out of your first session!).

You may access these servers over SSH, which is probably preinstalled on your device. For example, to access lnxsrv15 with the username eggert, use the following command: 

  ssh eggert@lnxsrv15.seas.ucla.edu

Please note that SEASnet is only available within the UCLA intranet. If you’d like to do this assignment while outside of UCLA, please consider using a VPN.

Now, run the following commands:

  cp -r ~eggert/cs35L/assign1 .
  cd assign1
  . ./helper init

This makes a copy of the working directory assign1 you’ll use to complete these assignments. It also makes some modifications to your system to work with this assignment. Make sure to run all commands in this assignment inside this directory. Each time you login to SEASnet, you must change your working directory to assign1 as shown above.

Laboratory: GNU/Linux and Emacs scavenger hunt

Lab 1.0: Getting started with GNU/Linux and Emacs

No submission is required for this lab section, but please do it to make sure your system is set up correctly and to familiarize yourself with Emacs.

  1. Make sure to have followed the steps in Setup.
  2. Run the shell command type emacs which should report that it is /usr/local/cs/bin/emacs.
  3. Run the shell command emacs --version which should report that it is version 30.1.
  4. Run the shell command emacs to enter a editing session.
  5. Type control-H followed by t ("C-h t", in Emacs-ese). This should drop you into the Emacs tutorial.
  6. Take the tutorial.

For the following lab exercises, use intelligent ways of answering the questions. For example, if asked to move to the first occurrence of the word "scrumptious", do not merely use cursor keys to move the cursor by hand; instead, use the builtin search capabilities to find "scrumptious" quickly.

Lab 1.1: Moving around in Emacs (10 points)

  1. Download a copy of the web page you’re looking at into a file named assign1.html. You can do this with the following shell command: 
    wget2 URL
    where URL is this web page’s URL.
  2. Use the shell command cp to make three copies of this file. Call the copies exer1.html, exer2.html, and exer3.html.
  3. Use Emacs to edit the file exer1.html.
  4. Before continuing, run the Emacs command M-x persist-drib lab1.1.drib to create a dribble file lab1.1.drib. This records everything you type for this lab section.
    • If you’d like to redo this lab section, exit Emacs and launch it again.
    • You must run the previous command each time you launch Emacs to record your keystrokes. This function will append to the dribble file, so don’t worry about overwriting it.
  6. (2 points) Use searching to move the cursor to just after the first occurrence of the string "UTF-8" (all upper-case).
  7. (2 points) Now move the cursor to the start of the first later occurrence of the word "scavenger".
  8. (2 points) Now move the cursor to the start of the first later occurrence of the word "self-referential".
  9. (2 points) Now move the cursor to the start of the first later occurrence of the word "arrow".
  10. (1 point) Now move the cursor to the end of the current line.
  11. (1 point) Now move the cursor to the beginning of the current line.
  12. Did you move the cursor using the arrow keys instead of using Emacs’s searching capabilities? If so, please learn how to search, and repeat the above steps by doing searching.
  13. When you are done, exit Emacs.

For this lab section (and the subsequent ones) do not worry about getting each keystroke exactly correct, but do try your best. The autograder looks for the most "significant" keystrokes in some subsequence (e.g., C-s scavenger) as proof you’ve attempted the lab.

Lab 1.2: Deleting text in Emacs (10 points)

  1. Use Emacs to edit the file exer2.html.
    • The idea is to delete its HTML comments; the resulting page should display the same text as the original.
    • Just like Lab 1.1, use the persist-drib command to create a dribble file lab1.2.drib.
    • (2 points) You must also submit exer2.html.
  2. (1 point) Delete the 107th line, which is an HTML comment. <!-- HTML comments look like this, but the comment you delete has different text inside. -->
  3. (1 point) Delete the HTML comment containing the text "DELETE-ME DELETE-ME DELETE-ME".
  4. (2 points) Delete the HTML comment containing the text "https://en.wikipedia.org/wiki/HTML_comment#Comments".
  5. (4 points) There are two more HTML comments; delete them too.

Once again, try to accomplish the tasks using a small number of keystrokes. When you are done, save the file and exit back to the command line. Also, check that you haven’t deleted something that you want to keep, by using the following shell command:

  diff -u exer1.html exer2.html >exer2.diff

The output file exer2.diff should describe only text that you wanted to remove. Don’t remove exer2.diff; you’ll need it later.

Lab 1.3: Inserting text in Emacs (15 points)

  1. Use Emacs to edit the file exer3.html.
    • Just like Lab 1.1, use the persist-drib command to create a dribble file lab1.3.drib.
    • (2 points) You must also submit exer3.html.
  2. (1 point) Change the first two instances of "Assignment 1" to "Assignment 27".
  3. (1 point) Change the first instance of "UTF-8" to "US-ASCII".
  4. (3 points) Ooops! The file is not ASCII so you need to fix the file so that it is ASCII. Most of its non-ASCII characters are the Unicode character “’” (RIGHT SINGLE QUOTATION MARK U+2019); fix these by replacing each one with an ASCII “'” (U+0027 APOSTROPHE). Use M-x replace-string to do this systematically.
  5. (3 points) Find the first remaining character in the file that is not ASCII. You can find the next non-ASCII character by searching for the regular expression "[^[:ascii:]]".
  6. What non-ASCII character is it? You can use C-u C-x = (what-cursor-position) to find out. (No need to submit; this is just for your own curiosity.)
  7. (5 points) Remove every line that contains a non-ASCII character other than the U+2019 characters you already replaced.
  8. When you finish, save the text file and exit Emacs. As before, use the diff command to check your work.

Lab 1.4: Other editing tasks in Emacs (20 points)

In addition to inserting and deleting text, there are other common tasks that you should know, like copy and paste, search and replace, and undo.

  1. Execute the shell command "cat exer2.html exer2.diff >exer4.html" to create a file exer4.html that contains a copy of exer2.html followed by a copy of exer2.diff.
  2. Use Emacs to edit the file exer4.html.
    • The idea is to edit the file so that it looks identical to exer1.html on a browser, but the file itself is a little bit different internally.
    • Just like Lab 1.1, use the persist-drib command to create a dribble file lab1.4.drib.
    • (2 points) You must also submit exer4.html.
  3. (4 points) Go to the end of the file. Copy the new lines in the last chunk of diff output, and paste them into the correct location earlier in the file.
  4. (4 points) Repeat the process, until the earlier part of the file is identical to what was in the original.
  5. (2 points) Delete the last part of the file, which contains the diff output.
  6. (1 point) … except we didn’t really want to do that, so undo the deletion.
  7. (2 points) Turn the diff output into a comment, by surrounding it with "<!--" and "-->". If the diff output itself contains end-comment markers "-->", escape them by replacing each such "-->" with "--&gt;".
  8. (5 points) Now let’s try some search and replaces. Search the text document for the pattern "<ol>". Use the search and replace function to replace them all with the final-caps equivalent "<oL>".
  9. Check your work by running the shell command "diff -u exer1.html exer4.html >exer4.diff". The only differences should be changes from "<ol>" to "<oL>", and a long HTML comment at the end.

Lab 1.5: Exploring the operating system outside Emacs (13 points)

There are several short questions to answer on Gradescope. Use the commands that you learned in class to find answers to these questions. Don’t use a search engine like Google to find previous editions of this assignment and/or its answers, and don’t ask your neighbor, don’t use GitHub, etc. When you find a new command, run it so you can see exactly how it works.

Lab 1.6: Doing commands in Emacs (7 points)

Do these tasks all within Emacs. Don’t use a shell subcommand if you can avoid it. (No need for a dribble file this section.)

  1. In the assign1 directory, there’s a C source file hello.c that contains the following text. Open it in Emacs.
    #include <stdio.h>
int
main (void)
{
  for (;;)
    {
      int c = getchar ();
      if (c < 0)
        {
          if (ferror (stdin))
            perror ("stdin");
          else
            fprintf (stderr, "EOF on input\n");
          return 1;
        }
      if (putchar (c) < 0 || (c == '\n' && fclose (stdout) != 0))
        {
          perror ("stdout");
          return 1;
        }
      if (c == '\n')
        return 0;
    }
}
  2. Compile this file, using the Emacs M-x compile command.
  3. (2 points) Run the compiled program from Emacs using the M-! command, and put the program’s standard output into a file named hello-a1 and its standard error into a file hello-a2.
  4. (2 points) Same as before, except run the program with standard input being closed, and put the program’s standard output and error into hello-b1 and hello-b2, respectively. Here, “closed” does not mean the standard input is an empty file; it means that standard input is not open at all, to any file. You can arrange for this via the <&- shell syntax.
  5. (2 points) Same as before, except run the program with standard input being the file /etc/passwd, and put the program’s standard output and error into hello-c1 and hello-c2.
  6. (1 point) Same as before, except run the program with standard input being the file /etc/passwd and standard output being the file /dev/full, and put the program’s standard error into hello-d2.

Homework: Scripting in the shell (25 points)

Shell scripting

Read enough about the shell to understand how to do I/O redirection and pipes.

For each shell command tr, sort, comm, read the command’s man page and use that to deduce what the command should do given its operands shown below.

Examine the SEASnet file /usr/share/dict/linux.words, which contains a list of English words, one per line. Each word consists of one or more ASCII characters.

Use sort with < and > to sort this file and put the sorted output into a file sorted.words.

Then, take a text file (e.g., assign1.html), and run the following shell commands with that text file being standard input. Also, look generally at what each command outputs (in particular, how its output differs from that of the previous command), and why.


tr -c 'A-Za-z' '[\n*]'
tr -cs 'A-Za-z' '[\n*]'
tr -cs 'A-Za-z' '[\n*]' | sort
tr -cs 'A-Za-z' '[\n*]' | sort -u
tr -cs 'A-Za-z' '[\n*]' | sort -u | comm - sorted.words
tr -cs 'A-Za-z' '[\n*]' | sort -u | comm -23 - sorted.words

Let’s take the last command as the crude implementation of an English spelling checker. This implementation mishandles the input file /usr/share/dict/linux.words! Verify this by running a shell command like this and inspecting the output:

tr -cs 'A-Za-z' '[\n*]' </usr/share/dict/linux.words |
    sort -u | comm -23 - sorted.words

Write a shell script named myspell that fixes this problem. Your script should read from standard input and write misspelled words to standard output, for a suitable definition of "word" that is broader than just a maximal sequence of ASCII alphabetic characters, but is no broader than need be. The shell command:

./myspell </usr/share/dict/linux.words

should output nothing, because the dictionary by definition contains only correctly-spelled words.

Autograder Test Cases

Common Problems

Submission Requirements

For Lab 1.5, remember to complete the written assignment in Gradescope.

Submit the following files to Gradescope within a ZIP file named assign.zip by the posted deadline.

You may also use the command ./helper zip to generate this ZIP file. It will run some basic checks (includes all requisite files, the dribble files are non-empty, myspell is at least partially functioning) before zipping your submission.

Feel free to run this command before completing the whole assignment; a partial ZIP file will be created. You may run this through the Gradescope autograder to see what score you’d get up to this point.

To transfer the ZIP file from SEASnet onto your local computer, feel free to use Secure Copy (SCP). For example:


scp eggert@lnxsrv15.seas.ucla.edu:~/assign1/assign.zip ~/Downloads
scp eggert@lnxsrv15.seas.ucla.edu:~/assign1/assign.zip %USERPROFILE%/Downloads # for Windows

will transfer the file to your Downloads folder.

You have unlimited submissions; we’ll choose your best scoring one. Make sure that you choose the best scoring one as your active submission. As per the syllabus, remember that all code is subject to manual inspection.