- Configured cluster network for 10gig networking
- Planned out future experiments
- Setup spark 2.0
- Attended class
- Created a script for automatically loading tmux workspace
- Planned future of SOCC Paper for next two months.
- Started new experiments
A General Note
I’m finally back in the hang of things. I have to apologize that my website postings have been non-existant lately since I was on internship this summer at IBM Research. Now that I am back in Amherst and have started the new semester, I hope to start up again with new weekly updates. Stay tuned!
Our SOCC paper was rejected this summer, as such we have to make some changes and find a new place to submit to. We have settled on Eurosys for now, which gives us a little under two months to pull things together. We decided to make the entire paper use 10gig networking rather than the half 1gig half 10gig setup we had earlier. This will make the paper a lot stronger and hopefully confuse less reviewers. We also met this week to discuss some revisions to the experiments. A few of the reviewers complained that our workloads are not representative of real world data centers. We completely agree and are working towards trying to find more realistic scenarios we can evaluate. We are going to try and run a more realistic spark workload and a third workload that you may find in production. We also decided to rerun all previous experiments so that htey represent a 10gig network. Hopefully we’ll be able to pull together a nice overview of MPTCP in the datacenter for Eurosys.
Classes have started again much to my chagrin. This semester I am taking two cores in hopes of making up for some screw ups last year. I settled on taking an introduction to Logic course and a course on Visual Analytics. Introduction to logic seems like it will be useful but as it is more math based I may struggle a bit. Hopefully I’ll be able to work through it though and actually pick up the material. Visual Analytics is something I am very interested in and excited to learn more about. My research does a lot of measurement based work and I always find it difficult deciding on the best way to present results. I am hoping visual analytics will teach me a thing or two about the right way to present large data sets. I’ll try to update my weekly logs with some of the stuff I’m learning as it applies to my work.
Part of my summer work was looking at scaling containers and measuring performance. It gave me some ideas on how I can improve my experimental process to be more efficient. I am currently thinking of building a system that allows me to launch monitoring agents that measure performance of the system and then automatically cleans and presents the data in a nice format. A lot of this is built off of my summer work but I am hoping to write it in a new way so that it is more robust and multi-purpose. I hope to eventually use this system when running my many experiments in the future. I’ll hopefully put up a full blog post on both how I run experiments and what this new system will do in a few weeks.
What’s up for next week?
- First homework assignments
- Finishing up re-running 10gig experiments
- Architecting new experiments to round out the paper.
- SEC ‘17 Slide for the October Conference.