Recovery of discovery

It's been a while, which let's me look at my last blog with perspective. I wish I could say that I have so much to say because I've been progressing so rapidly the last couple of weeks, but I can't. I got stuck for a couple of days, which ended up forcing me to think deeply about the problem and ways to solve it. Of course this isn't a bad thing. I ended up re-implementing my code to remove published services that hardly resembles the first solution. My last idea worked, but it was a bit of a hack. My new implementation uses avahi's built in "Item Remove" capabilities to announce a signal when a service is to be removed. Here's the catch, that signal only announces the name, stype, and domain, which is "HTTP Keyserver", "_geysign._tcp", and "local." Everyone that starts up Keysign will have the same strings for these variables, which begs the question as to how will avahi browser distinguish between remove signals from different users? There were two reasonable ways to address this issue. One would be to startup the server so that avahi publisher announces additional information that is uniquely identifiable to that computer. Unfortunately, the code require do this would have been ungainly and require repeating a lot of the same code already present. The second option was to add specific information to the name variable, which was what I ended up doing. As soon as Tobi gave me the okay, I was able to add this functionality to the code and viola, the program can now remove services when users either hit the back button or turn off the program. I experienced much joy upon seeing the appropriate service being removed from the list of discovered services in the logs.

However, as we know other situations could arise in which neither of those events occurred and someone's program ceased to publish services, such as a battery dying. Ideally, what would make sense is to have the browser continuously refresh its list of published services. It sounds simple enough and you would imagine that you could use Avahi in such a manner. After looking at several people's code in which they use Avahi discovery services and googling deep within the bowels of the internet, such a idea seems beyond the scope of Avahi. There's record of someone asking the same question and there are no responses to their question. Uh oh. However, I think Avahi is smarter than I previously thought it was. I thought the only way to remove services was by Avahi explicitly sending a remove signal. However, a few blurbs here and there have suggested that Avahi discovery services "know" when the signal is lost and initiates its own remove signal regardless if the program publishing services has completely and suddenly stopped running. I crudely tested it and even if I shut down the terminal that is running the publisher, my browser that is running in a separate terminal appears to still receive the remove signal. I just "discovered" this golden gem of knowledge about 20 minutes ago and I'm rejoicing that perhaps, yes perhaps, my fix is close to completion and will work as hoped!

I've also been working on a completely independent GUI to display the number of active users on the network, but I'll save that story for another time...

NEXT!

It seems like if someone was reading through this, it would be helpful if I stated the problem first and then the solution later. Right? Yesterday I sat at this very page, which was blank at that time, and desperately sought the words to clearly describe the potential steps to solve our question. That question is to alert the user to how many potential signees are on the currant network. The problem is two-fold. Currently, once someone announces their data, the system does not update their list of server-side clients. I could announce a key and, even though I've turned off the application, my data continues to reside in the clients list of discovered services. The next issue, which is much broader in scope, is that if I'm running the application in client mode I do not publish services. In effect I am invisible on the network. 

The sub-problem of revising the list of discovered services is almost an extension of my first task. Thus, it was the best place to start. Yesterday as I sat in front of a blank page, I realized that even though Tobi and I had gone through sketch of how one might go about implementing a solution, I didn't fully comprehend the problem. Even as I write this, I continue to think of things that may complicate or impact the solution. On a more general level, there are a few ways to tackle this problem. We can update the list by having Avahi announce that they are removing their services. Or, the browser can continually check for published data and revise the list by who is currently emitting a fingerprint. I went ahead an implemented a rough draft in which Avahi announces that they are removing their services. This "draft" is by no means ready to merge, since there are several smaller details and a few integral aspects that will need to be addressed. Briefly summarized, when the user clicks the back button to return to the original window of the application, the program tells Avahi to shut down the server. In my version, the server stops and then is re-initialized. I took advantage of the fact that the fingerprint in this case would be equal to None. When Avahi browser encounters a server-side client with a fingerprint set to None, it then removes that client's information from the on discovered list. It seems to work. I think a little tweaking and it should be okay. 

Once the feature is finished, I can compare the result to this post and reflect on the differences.

Success! First task is finished in GNOME Keysign

It feels good to be making progress. It is also a relief. When I first started, I have to admit that I had a little anxiety at the thought of accomplishing a "real," bigger contribution during the internship. The GNOME Keysign codebase isn't particularly large, but there are enough files to get lost. Recently, in an interview someone asked me to give an example of a design decision. Considering how important readability is to maintaining code, my initial perspective is that the logging has been invaluable. Python is fairly easy to read, but the logs helped me track the flow of data. If it wasn't for the logging, it would of taken me a lot longer to figure out the code base. After working with the logs and adding my own piece of functionality, I feel a lot more nimble within the source code. That is to say, I have a better idea where to look for something when searching for a piece of functionality. Yet, if I look back over the last week or so, it seems like another thing that really helped to get started was to just dive in and start messing with the code, even if it was just adding more print statements. It was at this point where I started making more and more progress each day, which is clearly reflected in my github commits.

Good thing that I just learned how to use the squash command in Github. Some of those commits will disappear forever!

Another challenge to the project, is the number of subfields that I will work with over the period of the internship. My first task involved working with Avahi browser. I didn't know anything about networking, which I thought would be a barrier. However, I guess that's what information hiding is; you don't really need to know the implementation of something to use it. Anyway, something to keep in mind when I come up against another unfamiliar concept.

On to the implementation. I left off with the server announcing the fingerprint in a readable format. I thought the solution would involve rejecting the server and not adding their information to the list of discovered services. However, the better implementation was to transfer the data to the GetKeySection where I filtered out all the published data that did not contain a matching key. After reviewing this version, Tobi suggested that instead of filtering the list of server data, that we should sort the published data by matching keys. If there is a matching key on the list, it will be sorted so that it is the first entry which will result in the application only downloading the information for that key. What we have is that if the key is not present in the list, then the program will download all information from all services discovered. I struggled a bit with why this was a preferable method until Tobi mentioned backwards compatibility. If someone has the old version of GNOME Keysign that does not announce a key, then the second version will not download the information even if it is the correct key. Thus, this is another reason why my original idea of not adding the service if it did not have a relevant key was not as sound as I had thought. It too would not maintain backwards compatibility. After a few more revisions to bring the code up to standard, everything is set and ready to merge.

As a side to this story, having my code merged is a very satisfying feeling, especially in the realm of open source. Some cynics may doubt me, but it gives me a genuine warm fuzzy feeling. :-)

Mini update

Progress! When I first started the task of working with Avahi to announce the fingerprint (mostly completed by AM) and then only download the information if the fingerprint is a match to the fingerprint to be signed, I have to say I wasn't very confident in finding a solution. It's seems that I am inching forward and getting closer.

The last few days I was able to get AM's code up and working. Now, I feel like I can answer that yes it is definitely working. As a test, I can run the server side with one fingerprint while entering different fingerprints in the client side and it does not appear to affect which fingerprint the server side announces. In addition, I was able to get the avahi.txt_array_to_string_array to display as something that resembles English. However, even though the command spells out exactly what it is doing, it took me a bit to realize that it was displayed as a string representation of a list including all the brackets, commas, and quotation marks. A simple for loop converted the string array into a regular eight character string representing the end of the fingerprint. I'm not sure this is the best way to go, but I feel like this is something that can be ironed out after the main functionality is met. With a few additional modifications I was able to get that piece of data into the on_new_service function of the MainWindow.py file. Why do I want this data here? My thought is that I can modify the verify_service function to check if the announced fingerprint matches the entered fingerprint of the GetKey section. I think this will work since if I manually add one fingerprint into the verify_service section and then choose a different fingerprint in the client side, it rejects the client side and turns off the key server before downloading any information. So that's where I am currently, trying to get the last eight characters of the fingerprint in the GetKeySection into the verify_service function of MainWindow. I think this should not be so hard, but my brain hurts a little. 

If only Woolly Lemurs could code themselves

It's been a couple of crazy weeks lately, but it seems like things are getting back on track.

After the last blog post, my next order of business was to get the client and server sides working properly on my machine (note this information should have been its own blog post a while ago), which wasn't too difficult. I did run into a problem in which xdg-email uses a grep command that is not compatible with the Thunderbird mail client and thus the attachment with the signed key information fails to be included in the email. Putting this aside, perhaps what was more challenging was to get a handle on the codebase. My plan of attack was to run both the client and server side and to use the logs to track the flow of information. It seemed a little overkill at the time, but I annotated the logs so that every time there was log entry I could see which function was called. I am so glad that I took the time to do this, it has been more helpful than I would have thought. Not only was it a great aid in understanding the source code, it has been a useful resource in my current task. The following is a link to the log when using the server side:

https://drive.google.com/openid=0Bzur_5tg85CmekwtSUx4UVBZLWM&authuser=1

With the next link showing the log for the client side:

https://drive.google.com/file/d/0Bzur_5tg85CmWnpqYTFtYUNRM3c/view?usp=sharing 

I know this might not be possible to do for other projects, but I would definitely do something similar such as a flow chart or anything to track where the program is running. Alas, onto the current week. 

I began working on my first task, which is to add some functionality to the avahi services so that avahi only attempts to download keys from servers that have the correct information. Currently, the data with the public key is transferred between computers via the local network, with the use of the Avahi library with python bindings. Avahi is a system to allow users to publish and discover services on their local network. My first impression with Avahi is that it seems like it's rather obstinate and working with it may be challenging. Fortunately, the mysterious Andrei Macavei wrote code so that the fingerprint would be announced in text form. I thought the pragmatic approach would be to make sure that this piece of code functioned as it should. When I got it up and running on a new branch it appeared to interfere with rendering the fingerprint as a QRCode. However, it only took a few modifications so that the application functioned as before. That was nice. My next step was to test if the fingerprint was being published. The answer is, I think so. : /  As it stands, there is a necessary bit of code in the AvahiPublisher file, avahi.string_array_to_txt_array (self.service_txt), that is crucial for the program to function. However, that self.service_txt is my fingerprint information, which I can add by itself and it appears as text in the logs for the AvahiBrowser implementation. I think this is good and I'm planning to move onto working on the client side of things. Thus, the very basic summary of the solution is that I need to write code that will recognize the published fingerprint and then proceed to download the information only if it is a match with the relevant fingerprint. So simple, right? It's a bit of a daunting task, but my mentor (Tobi) has provided some great links to code that has what we need. Somewhere the solution is in there. So tomorrow begins the process of actually writing that solution so that it works!