We will use the VM settings specified in the
Vagrantfile provided by us to run and test your program. The python packages we have pre-installed in your VM includes the following ones:
These are enough for you to implement this project. Please make sure your Python program is runnable under the environment that we provide you. You will get zero points if your program has external dependencies that prevent it from running in our VM setting.
Django # The django framework psycopg2 # The Postgres Python adapter textract # The python package used to extract text from pdf pytz # The python package to get datetime of specific timezone.
In this homework, all development and testing will be done in Linux using a VirtualBox virtual machine. You will use Postgres as the database backend and Django framework as the application front end. You do not need to know Django; we have already configured everything for you. We will use Python 2.7 with psycopg2 to implement all the databse APIs. Your task is to implement the APIs between the frontend and the database.
To get started, first thing that you need to do is to setup the VM software. We are going to download and install VirtualBox and Vagrant. VirtualBox is a general-purpose full virtualizer for x86 hardware, targeted at server, desktop and embedded use. And Vagrant is a toolkit for VirtualBox. Follow the following steps:
Vagrant will create a complete development environment in the VM for this assignment. You need to download the configuration files. Untar the file using the following command:
$ tar xvf cmupaper_vm.tar
Enter the directory
cmupaper_vm. There is only one file inside the directory named as
Vagrantfile. This is the configuration file that Vagrant uses to setup the virtual environment. Let's launch our VM and starts playing with it by the following steps. Make sure you have already installed VirtualBox and Vagrant on your local machine.
The last command will provision the new VM for you. Make sure you have network access during the provisioning. It will automatically install PostgrestSQL and setup the Python virtual environment needed to develop the website. This should take around 5-10 min.
$ cd cmupaper_vm # Enter the vm configuration directory $ vagrant up # Launch the VM
cmupaper_vmin the host machine (i.e., not inside of the VM).
$ vagrant ssh
Note that after shutdown, all your files are still stored in the VM. You can access them again if you boot up your VM by
$ vagrant halt
By doing this, you will lose the entire VM
$ vagrant destroy
You will lose all your data on the VM after doing this. Use this command if you mess up your VM environment.
$ vagrant reload
$ cd ./cmupaper_vm # Go to the vm directory in your localhost
$ vagrant up # Boot up your VM if you haven't done so
$ vagrant ssh $ Login to the VM
./cmupaper/under your home directory. It contains all the source code for our database application. This directory should have the following layout:
./cmupaper/ ./cmupaper/manage.py # The python script to start the web server ./cmupaper/simple_checker.py # A simple sanity checker provided by us to test your API implementations ./cmupaper/hw7proj/ # The configuration of the web application ./cmupaper/media/ # The directory storing all uploaded data from clients ./cmupaper/paper/ # The direcory containing all the source code of our project ./cmupaper/paper/funcsionts.py #This is the only python file you are going to modify ./cmupaper/paper/[otherfiles] # Other python source files and html files to run the application
Note that, all APIs betweent the web frontend and the database are defined in
./cmupaper/paper/functions.py. This is the only file you are going to modify.
Altough we have not implemented any database APis yet, the web site is runnable! Let's first try it out:
$ cd ./cmupaper # Go to the project directory inside the VM
If every thing goes well, you will see something like this indicating that your webserver is running correctly:
$ python manage.py runserver 0.0.0.0:8000 # Run the web server
$ python manage.py runserver 0.0.0.0:8000 Performing system checks... System check identified no issues (0 silenced). October 29, 2016 - 21:17:03 Django version 1.10.1, using settings 'hw7proj.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C.
Note that it's important to specify you IP address as
0.0.0.0 and port number as
8000. Otherwise, you can not access to the webserver inside the VM via the NAT configuration setup by Vagrant on your localhost.
Now that the webserver is running, you can use the web browser on your local to visit the website. Open any browser you like, access the website by
127.0.0.1:15415. If everything goes well, you will see a login pager. The reason why we use port
15415 instead of port
8000 is because with configured Vagrant's NAT setting to forward the traffic from port
15415 outside the VM to port
8000 inside of it.
At this point, the website is useless and the only thing you can do is typing random login messages. Now we are going to make it more useful by implementing all the database APIs defined in
./cmupaper/paper/functions.py. Note that, this file is the only file you need to modify in order to run the website. Now let's take a look it this file. Checkout hints here to help you transfer files between your VM and your localhost.
In this file, we have defined about 20 APIs for you. We have included detailed inline comments describing the input and return value for each API.
Generally, the web application will invoke these APIs by a database wrapper (defined in
./cmupaper/paper/database_wrapper.py). The database wrapper will establish a database connection, pass it to the API and close the connection after the API call. Therefore, you do not need to create and terminate the connection by yourself
Each API has the return value following the pattern of
(status, retval). While
status indicates whether the API call successes or not, the
retval is the actual return value of the API. Please read the inline comments of every API carefully follow the return value format.
As we have said, your task is to implement and test all of the APIs defined in
./cmupaper/paper/functions.py. We also provide a simple sanity checker. You can run it under the
./cmupaper directory by the following command:
It will check the basic correctness and output format of APIs you have implemented. Note that, it does not test for all corner cases. Even if you have passed all the sanity tests, you may still fail in our final test. We encourage you to develop your own test cases. Please checkout the hints to debug and test your program.
$ python simple_checker.py
Since we are using Postgres in a Python environment, we are going to use
psycopg2, a python adapter of Postgres, to connect our application to the backend. We have provided a simple example on how to use
./cmupaper/paper/functions.py. Checkout the official documentations and tutorials here
Note that, you have to use the schema provided by us. Otherwise, our auto graders may not be able to grade your program and give you 0 points.
After you have implemented the
reset_db() API, you can reset and install your databases by the following steps:
127.0.0.1:15415/resetand the web server will invoke your
A printed version of your
./cmupaper/paper/functions.py. Don't forget to include your name and andrewId in the header comment of the source file.
A tar file named
[andrew_id]_hw7_phase2.tar, it should contain the following things:
Again, Don't forget to include your name and andrewId in the header comment of the source file.
./functions.py # Your implementation of all the database APIs ./customize_checker.py # Your customized checker to test your implementation. ./ReadMe # A brief introduction on how to run your checker
The way our auto grader works it that it first launch exactly the same VM you use for this project and then run test scripts against your submitted files. Basically, what we will do is similar to the
simple_checker.py. However, we will perform more sophisticated tests to test against possible corner cases. We will also directly contact to the Postgres database to make sure you use the database correctly.
You can test your implementations by the following approaches:
Once you get Python exceptions from your program, or your program does not act as expected, you can debug in the following ways:
psqlconsole to check whether you have done the right thing in the database.
The local directory containing the
Vagrantfile used to launch your VM is always synced to the
/vagrant directory inside your VM. That is, whatever you put under your local directory
./cmupaper_vm will appear in the
/vagrant inside your VM and vice versa. Note that
/vagrant is under the root directory of your VM
In case you occasionally delete your VM or mess it up, please always keep a copy of your current progress in your local machine. We strongly suggest that you use
git to do version control.