- Established a new productivity workflow
- Received new workstation
- Ran increasing file transfer experiment
- Ran terasort at 240gb and 100gb
- Met with Don
- Setup Entrepreneurship List Serve
- HDFS Memory Mount Experiment
New Productivity Workflow
I regularly run into an awesome student in the CS Lab who is currently in the process of finishing her Phd while also balancing a start up. We always have some really fascinating discussions. This week we discussed the Getting Things Done methodology. I had previously heard of it but every time I tried it out, I dropped it really quickly.
It turns out I was doing it wrong and only scratching the surface of it! I decided to take a crack at it armed with a new tool to facilitate organization. Nirvanahq is a platform that allows you to implement the GTD methodology with minimal hassle. I have already seen an uptick in my productivity. I will provide more updates as the year progresses.
I received my new workstation this week! It’s a shiny 15” Macbook Pro w/ touchbar. This thing is amazing! The screen alone is a massive upgrade from my macbook air. I spent some time this week getting the machine all configured and organized. Unfortunately, I spent all Thursday building an emulator on my macbook air and when I went into work on Friday with only my new machine, all my code was stuck at home!
MPTCP File Transfer Experiment
It is commonly known that MPTCP performs poorly with short flows and as such we were curious to see at what point MPTCP overtakes TCP when transferring files. We performed a brief experiment (~not up to peer review standards!) where we ran file transfers of varying sizes between two mptcp machines. Our saw that around 100mb is when MPTCP overtakes TCP. This is likely because it is large enough that the time to transfer is longer than it takes to establish the subflows and exit slow-start. Unfortunately, this may be bad for us as the workload we run on spark performs many transfers of size 50mb so MPTCP is actually hampering performance slightly. We are going to try and perform and emulation of the workload to see if this is truly the case. Hopefully next week will have those results so we can make an informed decision on the type of workloads we want to focus on.
Entrepreneurship List Serve
Another student(the same student who introduced me to GTD) and I decided that we should try to encourage more work at CICS in fostering entrepreneurship so we decided to start an internal list serve for students to discuss such topics. I spent some time this week signing people up and maintaining said list. The hope is that students will see how cool their research is and how easy it can be to turn that into a company. Hopefully good things will come of this.
HDFS Memory Experiment
On the first read through of one of the papers we are using to guide our experiments with MPTCP I missed the fact that they mount half of their RAM as a tempfs for HDFS. They use this as the tmp location for HDFS to eliminate disk related bounds. I attempted to do something similar by running terasort on a smaller workload (100gb vs 240gb) since we only had 32gb to mount vs the papers 64gb. The results from these experiments were inconclusive as they were close to the non tmpfs results. This may be due to other optimizations the original paper performed that were not mentioned such as disabling swap or something. I am going to keep digging deeper to see what I can unearth.
This isn’t entirely research related, but I started doing yoga this week at my climbing gym. They offer a 6:45 am (holy cow batmam, that’s early!) Good Morning Vinyasa class free to members. It was amazing and very relaxing. It was a great way to start the morning and I hope it will serve to keep my head clear in the future so I can get more research done! I recommend other grad students give it a try, since most students seem to always be in a perpetual state of stress.
What’s up for next week?
- Run Shuffle Emulator
- TCP Probe w/ terasort
- Identiyfy new workloads with our desired properties
- AI Homework
- Candidate Friday Prep