Recovery of discovery

It's been a while, which let's me look at my last blog with perspective. I wish I could say that I have so much to say because I've been progressing so rapidly the last couple of weeks, but I can't. I got stuck for a couple of days, which ended up forcing me to think deeply about the problem and ways to solve it. Of course this isn't a bad thing. I ended up re-implementing my code to remove published services that hardly resembles the first solution. My last idea worked, but it was a bit of a hack. My new implementation uses avahi's built in "Item Remove" capabilities to announce a signal when a service is to be removed. Here's the catch, that signal only announces the name, stype, and domain, which is "HTTP Keyserver", "_geysign._tcp", and "local." Everyone that starts up Keysign will have the same strings for these variables, which begs the question as to how will avahi browser distinguish between remove signals from different users? There were two reasonable ways to address this issue. One would be to startup the server so that avahi publisher announces additional information that is uniquely identifiable to that computer. Unfortunately, the code require do this would have been ungainly and require repeating a lot of the same code already present. The second option was to add specific information to the name variable, which was what I ended up doing. As soon as Tobi gave me the okay, I was able to add this functionality to the code and viola, the program can now remove services when users either hit the back button or turn off the program. I experienced much joy upon seeing the appropriate service being removed from the list of discovered services in the logs.

However, as we know other situations could arise in which neither of those events occurred and someone's program ceased to publish services, such as a battery dying. Ideally, what would make sense is to have the browser continuously refresh its list of published services. It sounds simple enough and you would imagine that you could use Avahi in such a manner. After looking at several people's code in which they use Avahi discovery services and googling deep within the bowels of the internet, such a idea seems beyond the scope of Avahi. There's record of someone asking the same question and there are no responses to their question. Uh oh. However, I think Avahi is smarter than I previously thought it was. I thought the only way to remove services was by Avahi explicitly sending a remove signal. However, a few blurbs here and there have suggested that Avahi discovery services "know" when the signal is lost and initiates its own remove signal regardless if the program publishing services has completely and suddenly stopped running. I crudely tested it and even if I shut down the terminal that is running the publisher, my browser that is running in a separate terminal appears to still receive the remove signal. I just "discovered" this golden gem of knowledge about 20 minutes ago and I'm rejoicing that perhaps, yes perhaps, my fix is close to completion and will work as hoped!

I've also been working on a completely independent GUI to display the number of active users on the network, but I'll save that story for another time...

NEXT!

It seems like if someone was reading through this, it would be helpful if I stated the problem first and then the solution later. Right? Yesterday I sat at this very page, which was blank at that time, and desperately sought the words to clearly describe the potential steps to solve our question. That question is to alert the user to how many potential signees are on the currant network. The problem is two-fold. Currently, once someone announces their data, the system does not update their list of server-side clients. I could announce a key and, even though I've turned off the application, my data continues to reside in the clients list of discovered services. The next issue, which is much broader in scope, is that if I'm running the application in client mode I do not publish services. In effect I am invisible on the network. 

The sub-problem of revising the list of discovered services is almost an extension of my first task. Thus, it was the best place to start. Yesterday as I sat in front of a blank page, I realized that even though Tobi and I had gone through sketch of how one might go about implementing a solution, I didn't fully comprehend the problem. Even as I write this, I continue to think of things that may complicate or impact the solution. On a more general level, there are a few ways to tackle this problem. We can update the list by having Avahi announce that they are removing their services. Or, the browser can continually check for published data and revise the list by who is currently emitting a fingerprint. I went ahead an implemented a rough draft in which Avahi announces that they are removing their services. This "draft" is by no means ready to merge, since there are several smaller details and a few integral aspects that will need to be addressed. Briefly summarized, when the user clicks the back button to return to the original window of the application, the program tells Avahi to shut down the server. In my version, the server stops and then is re-initialized. I took advantage of the fact that the fingerprint in this case would be equal to None. When Avahi browser encounters a server-side client with a fingerprint set to None, it then removes that client's information from the on discovered list. It seems to work. I think a little tweaking and it should be okay. 

Once the feature is finished, I can compare the result to this post and reflect on the differences.

Success! First task is finished in GNOME Keysign

It feels good to be making progress. It is also a relief. When I first started, I have to admit that I had a little anxiety at the thought of accomplishing a "real," bigger contribution during the internship. The GNOME Keysign codebase isn't particularly large, but there are enough files to get lost. Recently, in an interview someone asked me to give an example of a design decision. Considering how important readability is to maintaining code, my initial perspective is that the logging has been invaluable. Python is fairly easy to read, but the logs helped me track the flow of data. If it wasn't for the logging, it would of taken me a lot longer to figure out the code base. After working with the logs and adding my own piece of functionality, I feel a lot more nimble within the source code. That is to say, I have a better idea where to look for something when searching for a piece of functionality. Yet, if I look back over the last week or so, it seems like another thing that really helped to get started was to just dive in and start messing with the code, even if it was just adding more print statements. It was at this point where I started making more and more progress each day, which is clearly reflected in my github commits.

Good thing that I just learned how to use the squash command in Github. Some of those commits will disappear forever!

Another challenge to the project, is the number of subfields that I will work with over the period of the internship. My first task involved working with Avahi browser. I didn't know anything about networking, which I thought would be a barrier. However, I guess that's what information hiding is; you don't really need to know the implementation of something to use it. Anyway, something to keep in mind when I come up against another unfamiliar concept.

On to the implementation. I left off with the server announcing the fingerprint in a readable format. I thought the solution would involve rejecting the server and not adding their information to the list of discovered services. However, the better implementation was to transfer the data to the GetKeySection where I filtered out all the published data that did not contain a matching key. After reviewing this version, Tobi suggested that instead of filtering the list of server data, that we should sort the published data by matching keys. If there is a matching key on the list, it will be sorted so that it is the first entry which will result in the application only downloading the information for that key. What we have is that if the key is not present in the list, then the program will download all information from all services discovered. I struggled a bit with why this was a preferable method until Tobi mentioned backwards compatibility. If someone has the old version of GNOME Keysign that does not announce a key, then the second version will not download the information even if it is the correct key. Thus, this is another reason why my original idea of not adding the service if it did not have a relevant key was not as sound as I had thought. It too would not maintain backwards compatibility. After a few more revisions to bring the code up to standard, everything is set and ready to merge.

As a side to this story, having my code merged is a very satisfying feeling, especially in the realm of open source. Some cynics may doubt me, but it gives me a genuine warm fuzzy feeling. :-)

Mini update

Progress! When I first started the task of working with Avahi to announce the fingerprint (mostly completed by AM) and then only download the information if the fingerprint is a match to the fingerprint to be signed, I have to say I wasn't very confident in finding a solution. It's seems that I am inching forward and getting closer.

The last few days I was able to get AM's code up and working. Now, I feel like I can answer that yes it is definitely working. As a test, I can run the server side with one fingerprint while entering different fingerprints in the client side and it does not appear to affect which fingerprint the server side announces. In addition, I was able to get the avahi.txt_array_to_string_array to display as something that resembles English. However, even though the command spells out exactly what it is doing, it took me a bit to realize that it was displayed as a string representation of a list including all the brackets, commas, and quotation marks. A simple for loop converted the string array into a regular eight character string representing the end of the fingerprint. I'm not sure this is the best way to go, but I feel like this is something that can be ironed out after the main functionality is met. With a few additional modifications I was able to get that piece of data into the on_new_service function of the MainWindow.py file. Why do I want this data here? My thought is that I can modify the verify_service function to check if the announced fingerprint matches the entered fingerprint of the GetKey section. I think this will work since if I manually add one fingerprint into the verify_service section and then choose a different fingerprint in the client side, it rejects the client side and turns off the key server before downloading any information. So that's where I am currently, trying to get the last eight characters of the fingerprint in the GetKeySection into the verify_service function of MainWindow. I think this should not be so hard, but my brain hurts a little. 

If only Woolly Lemurs could code themselves

It's been a couple of crazy weeks lately, but it seems like things are getting back on track.

After the last blog post, my next order of business was to get the client and server sides working properly on my machine (note this information should have been its own blog post a while ago), which wasn't too difficult. I did run into a problem in which xdg-email uses a grep command that is not compatible with the Thunderbird mail client and thus the attachment with the signed key information fails to be included in the email. Putting this aside, perhaps what was more challenging was to get a handle on the codebase. My plan of attack was to run both the client and server side and to use the logs to track the flow of information. It seemed a little overkill at the time, but I annotated the logs so that every time there was log entry I could see which function was called. I am so glad that I took the time to do this, it has been more helpful than I would have thought. Not only was it a great aid in understanding the source code, it has been a useful resource in my current task. The following is a link to the log when using the server side:

https://drive.google.com/openid=0Bzur_5tg85CmekwtSUx4UVBZLWM&authuser=1

With the next link showing the log for the client side:

https://drive.google.com/file/d/0Bzur_5tg85CmWnpqYTFtYUNRM3c/view?usp=sharing 

I know this might not be possible to do for other projects, but I would definitely do something similar such as a flow chart or anything to track where the program is running. Alas, onto the current week. 

I began working on my first task, which is to add some functionality to the avahi services so that avahi only attempts to download keys from servers that have the correct information. Currently, the data with the public key is transferred between computers via the local network, with the use of the Avahi library with python bindings. Avahi is a system to allow users to publish and discover services on their local network. My first impression with Avahi is that it seems like it's rather obstinate and working with it may be challenging. Fortunately, the mysterious Andrei Macavei wrote code so that the fingerprint would be announced in text form. I thought the pragmatic approach would be to make sure that this piece of code functioned as it should. When I got it up and running on a new branch it appeared to interfere with rendering the fingerprint as a QRCode. However, it only took a few modifications so that the application functioned as before. That was nice. My next step was to test if the fingerprint was being published. The answer is, I think so. : /  As it stands, there is a necessary bit of code in the AvahiPublisher file, avahi.string_array_to_txt_array (self.service_txt), that is crucial for the program to function. However, that self.service_txt is my fingerprint information, which I can add by itself and it appears as text in the logs for the AvahiBrowser implementation. I think this is good and I'm planning to move onto working on the client side of things. Thus, the very basic summary of the solution is that I need to write code that will recognize the published fingerprint and then proceed to download the information only if it is a match with the relevant fingerprint. So simple, right? It's a bit of a daunting task, but my mentor (Tobi) has provided some great links to code that has what we need. Somewhere the solution is in there. So tomorrow begins the process of actually writing that solution so that it works!

In the beginning...

The FOSS Outreach program has officially started!

I have to laugh a little about myself. A few days ago I was hoping that I could jump in and start doing something wonderful. As I started asking questions about the system, one of the first things I noticed was that I didn’t understand the product. Sure, the basic idea is GNOME Keysign is a tool to sign keys without the use of a key server. Back in October when I began learning about what this meant I had a simplistic view on OpenPGP. If you want to send a “secret” message you need to have a way to encrypt the cleartext to ciphertext and then back to cleartext. If you’re like me, you might start to imagine someone with a strong background in mathematics and cryptography drinking copious amounts of redbull and staying up all night fighting the war on cyber crime.  Yes, take that <insert current global enemy here>. Unfortunately this might make you want to slowly back away, but sending encrypted emails is much easier than you might imagine. The encryption algorithms run quietly behind the scenes while you’re blissfully unaware of the poor souls who probably spent their 20s working on cryptography. The “key” (hee haw!) to encryption is having the key necessary to lock and unlock the encryption. Simple enough, but how do you securely send this key to your friends? If they live near you could pedal your bike over and tell them what it is. However, this doesn’t work for people who live outside of your town or someone that you’ve never met. To solve this solution, you have an asymmetric keypair. One key is your private key that you never give out to anyone while the second is a public key that you can shout out to the world on the rough top while someone is filming for the next viral youtube hit. Okay, you probably don’t want to do that since part of protecting your data is conservatism, but the point is, you shouldn’t be able ascertain the private key from the public key. The neat thing about the keypair is that if one is used to encrypt a document and the other can decrypt that same document, you know that those two keys are part of the same keypair. This is great because it solves another problem; how do I know the encrypted document is sent by the correct person? Well, because the public key can only decrypt it if it’s part of the keypair for the private key. However, what if I’ve never met this person? Now we can start talking about keysigning and it’s advantages. My public key is part of a digital certificate that also includes certificate information, such as my user id, and one or more digital signatures. Others sign my key with a digital signature, which authenticates my public key. So if I know Adam and trust him, and Adam knows Betty and trusts her, ergo I can trust Betty. The more signatures I have on my public key, the more it can be “trusted.” All of this seams fairly reasonable and logical. After downloading the GnuPG package, I easily made myself a couple of keypairs and was off to protecting my personal data. Eazy Piezy.

One of the things that I failed to grasp was why. I pity the poor NSA analyst who has the misfortune to read any of my emails. They must be incredibly boring and riddled with aggravating grammatical mistakes. However, as mundane as they are, they are part of my data that I want to keep protected. I wouldn’t let officers inspect my house whenever they wanted. Whether you avidly follow the news or not, Snowden has become a household name, usually associated with the discovery that the government is encroaching on our civil liberties more and more. OpenPGP is a set of encryption protocols that gives the individual the ability to encrypt their documents with the aforementioned keypairs. I’m a bit of a lazy person and I can rationalize myself out of encrypting my meaningless one-line email, but I do want to insure that I have the choice to encrypt my data if I want. Here is the call to arms. If no one encrypts their emails or cares about encrypting their emails, it makes it that much easier for the government to take away this option. Here’s an analogy, most people I know have a password on their phone. Could you imagine the uproar if congress tried to initiate a bill that revoked this feature on phones? It wouldn’t happen. A good resource is written by Phil Zimmerman here, http://www.philzimmermann.com/EN/bibliography/bibliography.html.

The problem with the previous block of information is that it provides the background for GNOME Keysign, but it doesn’t tell us specifically why it does what it does and more of the technical details of the program. Again, Keysign is a tool to sign keys without the use of a keyserver. Why would you want to do that? Well, part of that answer is that the unique key fingerprint sequence of numbers and letters are overlapping between users, which can lead to downloading the wrong public key from the key server. Keysign circumvents this problem my downloading key information from a local network. When I start up Keysign I immediately see a user-friendly GUI with two tabs, Keys and GetKeys, which represents the client and server sides of the program. If I proceed with Keys as in I want someone to sign my key, Keysign fetches my fingerprint and encodes it in a QRcode image. You can choose to transfer your key fingerprint by transferring the actual sequence or by scanning the QRcode. Underneath the GUI the program is publishing my key information to the local network. If instead, I choose to GetKeys as in I want to sign someone’s key, I can either enter in the fingerprint manually or scan the QRcode. The program then grabs the key information from the local network and then adds it to my temporary keyring. It then signs it with my digital signature and sends this information back to the email associated with that public key to insure that the signature is going to the intended email of the person whose key I just signed.  Viola, signed keys!

Sadly, it took quite a bit for me to figure all of this out, which is why I felt like I needed to make light of myself for thinking that I would just, you know, start solving issues from the get-go. Lucky for me, the other participants of the program have expressed that they are all in a similar situation, which relieved some of my concerns about my own performance. Strength in numbers, right? Hopefully, my next post will involve a more profound inspection of the inner workings of the project. J

Transition

I should be focusing on something else at the moment (ahem), but I know how anti-climatic finishing a PhD is for many students. So, again I'm going to write about the moment so that maybe it will feel a bit more "official." On Tuesday I handed over my thesis to my committee and to the thesis office for preliminary review. I had to print out the one that I turned into the thesis office. There was something very satisfying about holding a physical representation of my dissertation in my hands. It was big, and I felt the size was a good representation of all the work that I've put into it the last couple of months. Also, I've never considered writing to be my strong suit, although for most people that's probably true, so it's kind of fun to realize that I CAN write a book if I wish. 

Along with finishing the dissertation, I've ordered the food, reserved the auditorium, printed off the official pages for my committee to sign, and set a date for my last day in the lab where I will hand off all my reagents and lab notebooks. After working on my thesis for so long, I can hardly believe that I will be done in a week. Time slowed down to a crawl while writing the thesis, only to speed up to it's maximum velocity in these last days as I finish up my graduate student duties. 

I wish I had something more profound to say to mark the occasion. It's more of a relief than anything. All I can say is: I can't express how much I'm looking forward to FINALLY making the complete transition to building my programming skills full time.