Comp 116

Undergraduate Research Symposium

Wed, 27 Apr 2011 10:31:22 EDT

This afternoon, 27 April 2011, at 3:30 in Sitterson 011 computer science undergraduates will present their research projects. It's your chance to see the kind of research opportunities available to you.

The program will feature:

H264Katana: Slicing and dicing H.264-compressed video streams Max Beckman-Harned, supervised by Ketan Mayer-Patel

Optimizing Ancestry Inference for Complex Pedigrees Abhishek Sarkar, supervised by Wei Wang

Browser Based Sonic Zoom: Using Javascript, HTML5, and Spatial Sound as Enabling Technology Cameron Swaim, supervised by Gary Bishop

Effect of Transient Cross Traffic on the RAPID Congestion Control Protocol Rebecca Lovewell, supervised by Jasleen Kaur

Low-Cost Robot Platform for Medical Robotics Research Pavel Chtcheprov, supervised by Ron Alterovitz

Hookt on Fon-iks Austin Matthews, supervised by Fabian Monrose

The Bathysphere: Motion Capture as Art Caitlyn Losee, supervised by Greg Welch

Modeling Essential Tremors (ET) Krishna Kollu, supervised by Greg Welch

Java Example

Wed, 27 Apr 2011 09:39:11 EDT

The next course in Computer Science is Comp 401 wherein you will learn Java. You already know how to program so learning Java will mostly be a matter of learning the way to say things in that language. Here is a brief example so we can talk about the key differences.

public class FindLargestSmallestNumber {
 
    public static void main(String[] args) {
 
        //array of 10 numbers
        int numbers[] = new int[]{32,43,53,54,32,65,63,98,43,23};
 
        //assign first element of an array to largest and smallest
        int smallest = numbers[0];
        int largest = numbers[0];
 
        for(int i=1; i< numbers.length; i++)
        {
            if(numbers[i] > largest)
                largest = numbers[i];
            else if (numbers[i] < smallest)
                smallest = numbers[i];
 
        }
 
        System.out.println("Largest Number is : " + largest);
        System.out.println("Smallest Number is : " + smallest);
    }
}
 
/*
Output of this program would be
Largest Number is : 98
Smallest Number is : 23
*/

In Python using numpy we could write it this way.

import numpy as np

numbers = np.array([32,43,53,54,32,65,63,98,43,23])

print 'Largest Number is :", numbers.max()
print 'Smallest Number is :", numbers.min()

That seems almost like cheating because its so easy. So let's look at the parallel example in pure Python.

numbers = [32,43,53,54,32,65,63,98,43,23]

smallest = numbers[0]
largest = numbers[0]

for number in numbers[1:]:
    if number < smallest:
        smallest = number;
    if number > largest:
        largest = number

print 'Largest Number is :", largest
print 'Smallest Number is :", smallest

Carolina Course Evaluation

Mon, 25 Apr 2011 09:11:50 EDT

Please complete the Carolina Course Evaluation form online.

Example Final Exam

Mon, 25 Apr 2011 08:00:56 EDT

Here is an example of the sorts of questions I may ask on the final exam. The final exam will not be limited to the scope of this sample. Anything we have discussed all semester is fair game. Of course, the material I emphasized is more likely to appear on the final.

1. The SouthEastern regional climate center,
http://www.dnr.state.sc.us/climate/sercc/, has historical weather data
from many weather stations. Here is some sample data for the Chapel
Hill station, including the maximum and minimum temperatures for each
day of the year, averaged over 30 years (1961 through 1990).

 CHAPEL HILL 2 W, NORTH CAROLINA 30 Year Daily Summary 
 CHAPEL HILL 2 W, NORTH CAROLINA 
30 Year Daily Temperature and Precipitation Summary 
 STATION 311677 AVERAGES FROM AVAILABLE YEARS IN PERIOD 1961 TO 1990 .

 DOY  MON DY  TMAX  #YRS  TMIN  #YRS PRECIP #YRS SD MAX SD MIN
    1  1  1   49.1  30.   26.9  30.  0.116  30. 10.895 10.164
    2  1  2   49.0  30.   26.8  30.  0.115  30. 10.868 10.169
    3  1  3   48.8  30.   26.7  30.  0.115  30. 10.864 10.104
•
•
•
  360 12 25   50.9  30.   28.4  30.  0.121  30. 10.816 10.073
  361 12 26   50.6  30.   28.2  29.  0.121  30. 10.799 10.042
  362 12 27   50.4  29.   28.0  30.  0.118  30. 10.831 10.013
  363 12 28   50.0  30.   27.7  30.  0.120  30. 10.831 10.056
  364 12 29   49.8  30.   27.5  30.  0.118  30. 10.878 10.154
  365 12 30   49.5  30.   27.3  30.  0.118  30. 10.961 10.219
  366 12 31   49.3  30.   27.1  30.  0.117  30. 10.939 10.161


a) Assume for these first 4 subparts that you have read the max and
min temperatures into column vectors TMAX and TMIN and the standard
deviations for max and min temperatures into column vectors SDMAX and
SDMIN. (Standard deviation is a statistic that tries to capture how
far we would expect to find the temperature from the day's max or
min. E.g., if max temperature is normally distributed, we expect that
90% of the days are less than 1.28 standard distributions above TMAX.)
Each vector has the same number of rows.

Write Python expressions that will do the following.

i. Plot the average temperature maxima (red) and minima (blue)

ii. Determine the highest maximum temperature.

iii. Determine which day (1 – 366) has the highest maximum temperature.

iv. Determine, for each day, the temperature range between max and
min.  (E.g., for day 360, which is Dec 25, the range is 22.5 degrees.)

b) Complete function tempExceptional which, given today's day number
(daynum) and temperature (temp), should determine if temp is within
the historical max/min range for that day, and if not, determine how
many standard deviations above the historical max for that day, or how
many standard deviations below the historical min for that day.

Set result = 0 for normal, result = a positive number of standard
deviations if above the max, or result = a negative number of standard
deviations if below the min.

def tempExceptional(daynum, temp, TMAX, TMIN, SDMAX, SDMIN):
    # daynum = the day number today (1 - 366) 
    # temp = the current temperature in degrees 
    # TMAX, TMIN, SDMAX, SDMIN column vectors as in a).


2. Write functions to solve at least 3 of the following 5 challenges.

a) Function twomin(x) should take a list of numbers in a vector x,
find the two smallest, and return them in a list result being careful
to properly handle duplicated items in the list.

b) Write a function dist(p,q) that takes a point p represented as a
1×d row vector, and a list of n points q, represented as an n×d
matrix, and returns an n×1 vector of the distance from point p to each
point q[i,:].

c) Write a function pairdist(p,q) that takes two matrices, say m×d and
n×d, and returns the m×n matrix of distances between all pairs of
rows. You may use the function dist() from b), even if you choose not
to write that one.

d) isPrime(n) returns True if the number N is prime, False otherwise.

e) isPalindrome(s) returns True if s is a palindrome (same read left
to right as right to left), False otherwise.

3. Multiple choice (circle the roman numerals for all that apply)

a) If A is a 4x6 matrix, in assignment A[2:3,4:5] = B; the value of B
can be
i. []
ii. scalar
iii. a 2x2 matrix
iv. a 4x1 matrix
v. a 4x6 matrix

b) If M is a 3x2 matrix, the value of M[:] is a 
i. error
ii. 3x2 matrix
iii. 2x3 matrix
iv. row vector 1x6
v. column vector 6x1

c)The differences between for and while loops include:

i. You will always use a colon : in a for loop
ii. A while loop can run forever, but a for loop cannot.
iii. A for loop defines a variable; a while loop may not even need a variable.  
iv. A for loop executes the indented statements at least once; a while loop 
may never execute the statements.

d) The differences between floating point and integer numbers include:

i. floats cannot be used as array indicies.
ii. integer arithmetic is exact, floating point is approximate.
iii. the exponentiation operator ** cannot be used on integers.
iv. the range function only works on integers.

4. Floating point numbers are unlike the real numbers we study in
mathematics. They approximate the behavior of real numbers reasonably
well but they differ in important ways that mostly result from their
limited precision. For example X+1 == X is never true for real numbers
but is sometimes true for floating point. Show a fragment of Python
code that finds the smallest X such that X+1 == X.

5. The following 3 mystery functions inefficiently implement methods
that are provided by numpy arrays. Give me the names of the
corresponding functions from numpy.

def F1(A):
    if len(A) == 1:
        return A[0]
    else:
        t = F1(A[1:])
        if t > A[0]:
            return t
        else:
            return A[0]

def F2(A):
    t = 0
    for x in A:
        t += x
    return t

def F3(A):
    s = 0
    for x in A:
        s += 1
    return s

6. Show how to create the following arrays.

a. array([[ 0,  1,  2,  3],
          [ 4,  5,  6,  7],
          [ 8,  9, 10, 11]])

b. array([[ 0,  4,  8],
          [ 1,  5,  9],
          [ 2,  6, 10],
          [ 3,  7, 11]])

c. array([0, 1, 4, 9, 16, ..., n**2]) for any positive integer value of N

d. A 12x12 checkerboard pattern with alternating 0's and 1's. Like
this 4x4 version only bigger.

[ [ 0, 1, 0, 1],
  [ 1, 0, 1, 0],
  [ 0, 1, 0, 1],
  [ 1, 0, 1, 0] ]

e. array([[ 0.,  0.,  0.,  0.,  0.],
          [ 0.,  1.,  1.,  1.,  0.],
          [ 0.,  1.,  0.,  1.,  0.],
          [ 0.,  1.,  1.,  1.,  0.],
          [ 0.,  0.,  0.,  0.,  0.]])


7. Suppose you are given an array named W with the birth weight of N babies.
Use numpy array methods to find the following.

a. the weight of the heaviest baby.

b. the mean weight of all the babies.

c. the median weight for all the babies.

d. the standard deviation of the weights.

e. the mean of the weight of babies that were heavier than 3 pounds.


8. Mark the following statements with the data types they produce. The
first is done for you as an example. Mark X for error, otherwise mark
with an uppercase letter for the basic type and a lowercase letter for
the shape.

F float
I integer
B boolean
S string

s scalar
v vector
m matrix

X error

Assume these values for the variables: x = np.arange(4.0); y = x**2; z = np.eye(4);

F  v  a) x+y
__ __ b) y.sum()
__ __ c) z*z
__ __ d) np.dot(x,z)
__ __ e) x == np.sqrt(y)
__ __ f) y < 5
__ __ g) y[x>2]
__ __ h) float('3.1415')
__ __ i) str(1.5)

9. What are the differences between the return statement and the print statement?

10. A newbie programmer has written the following code and is
surprised that it doesn't terminate after printing the 10 lines but
just keeps on printing ever larger numbers. Explain what is going
wrong.

i = 0
while i != 1:
    i = i + 0.1
    print i

Analysis Example

Thu, 14 Apr 2011 08:02:51 EDT

Here is a complete example of an analysis script. The data file consists of lines like these with the prompt to type, the hand to use, and the time it took separated by spaces. The lines look like this:

xbafkn 0 1.2505370144
ugqrsmx 1 1.37932912815
jegzf 0 1.09309074306
hrpi 1 0.842401438712
bezud 0 1.10971105873

These data are stored in a folder named a7data in files with names like subjaa, subjab, etc.

This script shows how I might read those data, analyze them, and produce a simple graph.

import numpy as np
import pylab
import os

lengths = [] # lengths of each prompt
hands = []   # hand used to type it
times = []   # time required

# os.listdir will return a list of the names of the files in a folder
# for each filename in the folder
for fname in os.listdir('a7data'):
    # for mac users
    if fname.startswith('.'):
        continue
    # open the file for reading
    fp = file('a7data/' + fname, 'r')
    # for each line in the file
    for line in fp:
        # split it into the three fields
        prompt, hand, time = line.split()
        # convert the hand to an int
        hand = int(hand)
        # and the time to a float
        time = float(time)
        # I'm only interested in the length of the prompt for my analysis
        length = len(prompt)
        # append them to the lists
        lengths.append(length)
        hands.append(hand)
        times.append(time)
# make numpy arrays out of the lists
lengths = np.array(lengths)
hands = np.array(hands)
times = np.array(times)

# create Boolean arrays for the hands
left = hands == 0
right = hands == 1

# plot the times for left and right as points
pylab.plot(lengths[left], times[left], 'bo',
    lengths[right], times[right], 'ro')

# fit a line equation to all the data, should have done the left and right separately
A = np.array([lengths, np.ones_like(lengths)]).T
B = np.array([times]).T

soln = np.linalg.lstsq(A,B)
X = soln[0]

# plot the best fit line
R = np.arange(3,8)
pylab.plot(R, X[0]*R + X[1], 'r')

# print the coefficients of the best fit line
print X

pylab.show()

It has come to my attention that some of you wrote lists of tuples to your data files. These files are pretty hard to take apart yourself but python has some help for you. The eval function will take a string and treat it like python source code. To make it more concrete, suppose my file is named data.txt and the content looks like this:

[('cow',1.34),('dog',1.5)]

I could read it like this:


# open the file, of course I could do this in a loop as in the above example
fp = file('data.txt', 'r')
# read all the content of the file into a string variable named bytes
bytes = fp.read()
# convert that string into a python list
data = eval(bytes)

Now data is a list of tuples of strings and floats.

Root Finding

Mon, 11 Apr 2011 12:30:00 EDT

Today we talked about finding the zeros (roots) of a function.

We used this function as an example.

def f(x):
    return np.sin(10*x) - x**3 + 1

In class we came up with a function like this:

def trapZero(f, x0, x1):
    if f(x0) > 0:
        x0,x1 = x1,x0 # swap so f(x0) is the negative one
    while abs(x0 - x1) > 1e-6:
        xm = (x0 + x1) / 2.0 # compute the midpoint
        fm = f(xm) # evaluate the function at the midpoint
        if np.sign(fm) == -1: # close in on the root from the negative side
            x0 = xm
        elif np.sign(fm) == 1: # close in on the root from the positive side
            x1 = xm
        elif np.sign(fm) == 0: # we happened to hit it (very rare)
            return xm
    return (x0 + x1) / 2.0 # why didn't I return xm?

Quiz Prep

Mon, 11 Apr 2011 09:35:37 EDT

Practice for this week's quiz and for the final exam.

Write functions to solve at least 3 of the following 5 challenges.

Function twomin(x) should take a vector x of numbers, find the two smallest, and return them in a list result being careful to properly handle duplicated items. For example twomin(np.array([1,2,1,3,4]) should return [1,1].
Function dist(p,Q) that takes a point p represented as a 1×d row vector, and a list of n points Q, represented as an n×d matrix, and returns an n×1 vector of the distance from point p to each point q[i,:].
Function pairdist(p,q) that takes two matrices, say m×d and n×d, and returns the m×n matrix of distances between all pairs of rows.
Function isPrime(n) returns True if the number N is prime, False otherwise.
Function isPalindrome(s) returns True if the string s is a palindrome (same read left to right as right to left), False otherwise.

Assignment 7 Helps

Mon, 04 Apr 2011 12:11:18 EDT

Here are some short code fragments to help you with Assignment 7.

Write data to a file.

# get the subject id
subj = raw_input('Enter subject id: ')

# construct a name for this data set
file_name = 'data/' + subj + '.txt'
# create a file object with the given name
fp = file(file_name, 'w')

# data collection code goes here
for i in range(5):
    word = raw_input('enter word: ')
    # suppose I wanted to record the number and the word
    print >>fp, str(i) + '|' + word

fp.close() # close the file object and make sure all data is written

Make a prompt appear and disappear after a short time.

import sys
import time

def promptThenDisappear(prompt, timeToDisplay):
    # write the prompt
    sys.stdout.write(prompt)
    # make sure the user can see it
    sys.stdout.flush()
    # wait for the specified time
    time.sleep(timeToDisplay)
    # erase it
    sys.stdout.write('\r' + ' '*len(prompt) + '\n')
    sys.stdout.flush()

# example of using it

promptThenDisappear('test', 2)

Compare two strings and get a score from 0 to 1 for their similarity.

import difflib
def stringSimilarity(a, b):
    sm = difflib.SequenceMatcher(None, a, b)
    return sm.ratio()

Change the color of text in the console.

import os
import sys

if os.name == 'nt':
    # windows version
    colors = {
        'black': 0,
        'blue': 1,
        'green': 2,
        'cyan': 3,
        'red': 4,
        'magenta': 5,
        'yellow': 6,
        'white': 7
    }
    import ctypes
    handle = ctypes.windll.kernel32.GetStdHandle(-11)
    def setColor(textColor, backgroundColor):
        color = colors[backgroundColor]<<4 | colors[textColor] | 0x88
        r = ctypes.windll.kernel32.SetConsoleTextAttribute(handle, color)

else:        
    # non windows version
    colors = {
        'black': 0,
        'red': 1,
        'green': 2,
        'yellow': 3,
        'blue': 4,
        'magenta': 5,
        'cyan': 6,
        'white': 7
    }
    def setColor(textColor, backgroundColor):
        color = '\x1b[3%d;4%d;1m' % (colors[textColor], colors[backgroundColor])
        sys.stdout.write(color)

# examples of using it

# setColor('red', 'green')
# print 'this is red text on green background'

Display a picture for a short time.

import numpy as np
import pylab

# read the image. You could read several of course. You'll need to change the name below
pic = pylab.imread('mypic.jpg')

# create a black image the same size to use for hiding
black = np.zeros_like(pic)

# turn on interactive mode so the plot will update while the script is running
pylab.ion()

# show the picture
pylab.imshow(pic)

# wait for 1/2 second. Change the value of timeout to increase or decrease the time
pylab.ginput(timeout=0.5)

# display the black image instead
pylab.imshow(black)

# force the black image to display
pylab.draw()

Quiz Prep

Mon, 04 Apr 2011 09:34:38 EDT

Here are some programming challenges to get you ready for the Quiz on Wednesday 6 April. None of the problems on the quiz will be this hard but you may see problems like this on the final exam.

1. Write an expression to compute the sum of the integers from -5 through 15 inclusive using numpy.





2. Write a loop to compute the sum of the integers from -5 through 15 inclusive without using numpy. 




3. Given a numpy vector V, subtract its mean and divide by its standard deviation leaving the result in R. 




4. What famous constant is this code approximating? (try it at the command prompt then work out what it is doing).

import numpy as np

xy = np.random.rand(10000000, 2) * 2 - 1
xy2 = xy ** 2
z = xy2.sum(1)
count = np.sum(z < 1)
est = 4.0 * count / 1e7
print 'estimate is', est

Check your grades

Mon, 04 Apr 2011 08:26:09 EDT

Check your grades on BlackBoard. If something is missing that you submitted you need to get it straight with the teaching assistants now.