Subjective Testing

Tonight, the majority of the subjective testing was performed in order to determine whether an average listener was able to discern a positive difference between the sound quality of the Elac speaker when it was placed on the engineered stand.

A set of identical bookshelves was used as a control for the testing. These bookshelves were of standard issue to a dorm room, and their sturdy wood construction simulated an option that many individuals might use in absence of a speaker stand. In addition they measured 24" tall, only a 1.5" difference in height in comparison to height (a negligible height difference that a listener would not be able to detect).

The setup of the experiment was as follows: The subject was seated in a chair 6 feet directly in front of the speakers, facing the operation as to hinder their bias. In addition, they were told to cover their ears initally to prevent them from hearing which surface (the soft rubber or the hard wood) the speakers were placed on to further blind them. They were not told which stand configuration would be played first. They then listened to one track, closed their ears while the speakers were moved to their second configuration. After hearing the second track, they were asked to choose their preference, reasons for their choice, and their degree of certainty in their ability to differentiation (on a 1-10 scale). The following pictures show the setup used in the experiment.

This was performed twice. The first song played was Enya's "Caribbean Blue". This song was chosen on multiple recommendations from sources for its merit as a audio testing song. This is because of its high frequency range, and for its range in instrumentation. The second song played was of the user's choice. Many sources claimed that differentiation is most easily possible in a song that the user is intimately familiar with, as they are more aware of its subtle nuances.

After polling a sample of subjects, the following data was accrued. In total, a minimum of 15 subjects will be polled, but at this majority stage of the experiment the following shows preliminary results:

Sound Data Analysis

Data Compilation

Vibration data recorded form the Seismometer App was exported to Excel, summed, averaged, and compiled into a table. The vibrations on each axis for each recording were summed and then divided by the number of data points. The resulting averages were then converted into readable charts by song to show results visually. Below, we give the charts including the results for "Hotel California" and "Heart of a City (Ain't No Love)".

Meanwhile, the audio recordings were edited. This was done to ensure that any difference between the recordings would be caused solely by the isolation systems used. First, the original tracks were cut to forty seconds. Next the recording in between the start of recording and the start of the track and the time after the end of the recording and when the recording stopped were removed. The recordings were then timeshifted to closely match that of the original track. This was done to eliminate all factors other than a difference in the replicated audio itself that could lead to variable differences between the recordings and thus lead to in correct results.
Once this was completed, each audio file was separated into its own folder, since Similarity must compare two folders. The folders were organized for ease of access.
The audio comparison could now start. Each recording was compared to its respective original track. The percent content, tags, and precision were recorded for each. The data was organized in Microsoft Excel by track and isolation system. An average percent content was taken for each song and the individual recording percent content deviation from the average was made for each recording. The deviation from average for each isolation system in all 3 songs was then summed. The isolation systems will be judged using these numbers. The results for each isolation system are given below.

Isolation Testing Performed

Isolation testing was done at ExCite Center from 2-5 p.m. Vibrations were again measured using the Seismometer App. The two songs which featured heavier bass ("Hotel California" and "Heart of a City") were used for vibration testing. Testing was again done by placing an iPhone 5c on the base of the stand, and recording the vibrations detected while a track was playing. This time, the best design would be indicated by the least amount of vibration measured. This is because the isolation system, as its name indicates, is suppose to mechanically isolate the speaker from the stand. By definition, this meas that the best isolation system would absorb the most vibrational energy, thus giving the lowest vibrational reading at the bottom of the stand.
After this, the audio the speakers produced with each isolation system on the stand was recorded. This was measured with the Audio Recorder app on a Droid Turbo. The experiment was set up with the phone placed on an identical stand, centered on the edge nearest to the speaker of. The nearest edge stand itself was 2' from the forward-facing edge of the stand with the speaker on it, Forty seconds of each song was recorded as a 705 kbps .wav file for each isolation system. This will hopefully give some insight into which isolation system reproduced the audio most accurately.

Week 8 Lab Activitites

This week in lab, we concluded our preparation for the testing, which will be completed this week. The following tasks were completed, both regarding the testing and the project as a whole.

1. Review Project Scope/Timeline: As the term comes to a close (there are only 2 weeks remaining to complete the project), adherence to the project timeline will be crucial to ensure completion.

2. Review of Project Final Draft: The draft of our final report was returned to us and Dr. Knight's critiques were reviewed. This process will help us not only for this project, but for any amount of technical writing which will be done in the future.

3. Discussion of Testing Methods: In order to properly complete the project, there are two rounds of testing that must be performed. First isolation testing must be performed to determine the best orientation of the isolation materials that effectively dissipate external vibrations and remove them from the speaker cabinet. This will be comprised of two separate factors: vibration testing using the Seismometer application and a sound analysis using the Similariy application. High-quality, FLAC versions of 3 songs that have a dynamic range of frequencies will be chosen in an attempt to be able to discern a difference in the resulting sound quality.

Once these values are accrued, the group will be able to choose a final stand/isolation design to use in subjective testing. While this segment may be perhaps less scientific, it is arguably the most important component of the project (as the goal was to produce a stand that improved the listening experience for a consumer at a lower price than those commercially available). A test track that we believe results in a noticeable increase in sound quality will be chosen and shown to our test subjects. In addition, prior research has discerned that often tracks that individuals are most familiar with will be more prone to being analyzed with deep scrutiny. Therefore, we will ask our subjects to choose a song that they know intimately to see whether they were able to discern a difference.

4. Final Report and Presentation Preparation:  While testing needs to still be completed, some discussion on the timing and logistics in preparing the final deliverables was included in the lab activities so we were able to keep the final objectives very much in mind.

More Testing Considerations

We have chosen the tracklist that will be used to test our isolative materials and subjective testing. It is as follows:
     1) Hotel California - Eagles - Hell Freezes Over
     2) Caribbean Blue - Enya - Shepard Moons
     3) Heart of a City (Ain't No Love) - Jay Z - The Blueprint

Lastly, we have chosen the software that will be used to test the isolation system and the audio quality. The software is listed below:
     1) Seismometer App - measures vibrations as a deviation from the mean (mean = no vibrations)
     2) Audio Recorder - records sound files at 705 kbps

Testing Logistics

We contacted Sarah from the ExCITe Center to schedule our final tests, and are waiting for a response. In the next few posts, we will describe our final test method in detail and provide reasoning for each part of it.

Isolation Materials Cut to Size

Today, the group focused on the first step of building the isolation unit, cutting the materials to size. During this, we ran into some trouble neatly cutting the material to size. The sponges proved to be difficult to cut evenly. Ultimately, they ended up being slightly uneven and rough on the edge they were cut from. However, this should have negligible effects, if any, on the performance of the stand. There was also a slight overhang in the foam, about a millimeter at most. This should have no affect of the quality of the stand, as the base is covered fully, and vibration at the base of the speaker will be absorbed just the same. These slight errors only caused cosmetic damage, which can be fixed later in our final isolation unit.

Week 7 Lab Activities

With the final report drafted, the reality of the quickly approaching end of the term has officially sunk in. That said, we have adhered relatively well to the timeline that we set out in the beginning of the term, and are still on schedule to finish the project comfortably within the deadline. It remains paramount that we give the project near-daily attention in order to make this a reality.

This week in lab, we worked on completing several critical tasks that remain in the project. This includes:

1. Review of Project Timeline: The timeline was once again reviewed among the group members in order to confirm our progress. This entailed a review of the remaining tasks at hand (isolation components, testing) and choosing appropriate date ranges when these will be able to take place. 

2. Discussion of Isolation Design: The primary remaining task is the compilation of the various isolation components (rubber, cork, foam, felt) into an effective system for hysteric damping. These components have already been purchased, and all that remains is their assembly. Within the next few days, final methods for this assembly will be finalized, and this progress will be posted  on the blog as necessary.

3. Discussion of Testing Logistics: The other critical task that remains is the objective data acquisition and the subjective listening tests, which will function as our barometer for how well the stands ultimately performed. Possible dates and procedures for doing so were discussed, and these activities will commence in the coming weeks.

4. Start of Project Presentation: The guidelines and rubric for the final presentation were consulted. This presentation was started today (adding the theory, timeline, budget, and other information that is not dependent on future work) in order to save us time and anxiety in the future, which can instead be directed on the actual delivery of the presentation itself.

Final Report Draft

The final report was drafted. It was a compilation of the term's ideas and progress. This allowed us to look upon our progress and determine what we could have done better over the past few weeks, which directly correlated to what we will do in the next few weeks. Primarily, we noticed the variations in our data, which we will deal with in the future by testing more carefully and running more trials to ensure the the trends we saw are valid. However, while reviewing our sources, we did notice several parallels between what we found through testing and what the documents said. This was an encouraging thing to see, as it shows that our results, and therefore our project and how we did it, are valid.