My take on this thing called life!!

Moving over to Wordpress.com

2006-02-01T19:30:00.000+05:30

I am moving my blog over to WordPress.com. Blogger is nice when you are dipping your toes in blogging waters but you soon find that you need more than just a life jacket. You need scuba diving gear if you want to go deep. A nice comparison of Blogger and WordPress.com is here.
This is the direct link for my new RSS feed.

How to fix iPod Eq distortion.

2005-11-08T18:36:00.000+05:30

The Problem:
The iPod gives a somewhat harsh and metallic sound when the equalizer is turned on. It does not matter which preset you choose (except Flat). If you don’t keep the equalizer off, the sound comes out distorted and is not pleasant to listen to. It happens on all the iPod models, even on the latest 5G one which has the best sound among hard disc based models. The 5G fixes lot of problems with the earlier iPods like Noise Defect and comes with a number of improvements, especially in bass performance but the sound still clips with the equalizer.

The Reason:
There is nothing wrong with iPod. Really.

It has as good a sound as any other player. Though it sounds a little bright, the sound has lot of detail and really insignificant amount of distortion. Even the equalizer is well designed and behaves as it is supposed to. But that metallic sound? That clipping? Well, it’s a classic case of “garbage in garbage out”.

It’s those damn mp3s. Or rather the original CDs themselves.

In a race to sound louder and louder, the CD mastering engineers push the recording level to its limits. This is especially true for mainstream pop/rock music. Somehow the producers think that the CDs have to be really loud to make a better impression.

Now when these hot mastered CDs (or mp3s made from them) are played with the equalizer on, the total loudness level goes beyond what iPod can produce. Some of my mp3s had a loudness level of 99.6db!! Yikes! What can the poor iPod do when presented with this crap? Say I choose Dance preset which would typically apply a boost of 6 db. Add this to 97db (a typical figure) and you get a level of 103 db. Most players have an upper limit of about 95db, some even going till 100db but that’s it. No wonder the sound comes out distorted and harsh.

The Fix:
All we need to do is bring down the loudness level of the mp3s down so that we get a little headroom to apply the Eq. The generally agreed upon loudness level for this is 89db.

Download mp3gain. It’s an open source freeware program. Add the folder containing the mp3s. Choose ‘track gain’ and click on the ‘track analysis’. It will calculate and display the loudness level for each song and also how much correction needs to be applied. When this analysis is done, just hit ‘track gain’ and it will apply the required correction to each song.
Mp3gain does not re-encode or otherwise modify/degrade the file in anyway. All it does is set a flag in the file. When a player reads this flag, it knows how loud to play this song.

This whole correction process is a little slow. It took about one hour per GB on my Pentium 4 machine, 30 hours total for my entire collection. It’s the analysis part that is slow. The correction is instantaneous. So instead of hitting the analysis and waiting for 30 hours to do the correction, a better way to do this is to hit the track gain directly. It will analyze and correct in one step.

After all this is done, erase all the songs from your iPod, resync, and enjoy!

Notes:

You can directly point mp3gain to iPod_control folder on the iPod but it is not recommended. You don't want that tiny hard disc on your iPod to be spinning continously for 30 hours. Moreover, if you do it on the PC, other mp3 players can also use this info.
You can stop/cancel the track gain process anytime you want. Mp3gain will pick up from where it left when you start it next time.
Mp3gain is more accurate at doing normalization than SoundCheck feature found in iTunes/iPod. Mp3gain is based on ReplayGain standard which takes into account the mechanism of how humans perceive loudness. Turns out that human ears use average energy over time to perceive how loud a certain sound is. So mp3gain divides each file into 50ms blocks and calculates the RMS energy value of each of these blocks. These values are then used to arrive at the overall RMS energy of the entire song. This is the value which is then used to normalize the song file.
SoundCheck works in similar way but it seems to take much longer blocks (fewer samples) to calculate the average RMS energy. This makes it less accurate (but a lot faster) than mp3gain.

Best values for digital camera settings!

2005-08-12T21:31:00.000+05:30

Back in the days of film, I used to own a Yashica MF-2 camera. It was as simple as a camera could get. There were no menus to fiddle with, no knobs to turn, and just one button to release the shutter. It didn't even need the batteries if you didn't want flash. And I should mention, it has never given me a technically bad shot - ever. All you had to do was press the button. How could you go wrong with that?

Then came the digital cameras and the question became - how could you ever get it right? My digital camera, an Olympys C-760, has more than 50 different settings! As Scot Adams said in Dilbert - we have come from simple tools like pointy wooden sticks to convolulted things like computers but our brains have not evolved at the same rate. And looking at my digital camera, I can personally vouch for that.

There are two type of settings in a digital camera. Ones that are same for each shot and ones that vary from shot to shot. Image quality (jpeg compression) is a fixed type of setting. You don't want a lower quality for one shot and higher quality for another, assuming you don't have any kind of strange fetish related to images. White Balance, on the other hand, has to be changed from shot to shot. Actually I don't change it that often but all the pros say that they do. So there must be something to it. Here is what I do for fixed settings:

Sharpening: Keep it 'Off'. If your camera doesn't have an explicit 'Off' setting, keep it to a minimum. I always used to keep it at 0, thinking that this would turn the sharpening off. It was only later I found that the scale was actually from -5 to +5. There is a reason people recommend to RTFM. A few reasons why in-camera sharpening is bad -

Different photos need different type and differnt amount of sharpening. Digital cameras usually employ Unsharp Masking but the only thing that you can vary in that is the amount, not the radius, or the threshold. And that's where the trouble lies. A tree shot with lots of leaves needs lower threshold than a Facial close up. Even if you could vary those other parameters, it would be too tedious to adjust it for shot to shot. So just turn it off and use your judgement later instead of leaving it to the camera.
The sharpening should always be assessed at 100% size. Those tiny 2" LCDs just don't cut it. And you have to fiddle with the sliders a lot before you get the look you want.
USM is not the best technique to sharpen photos. There are lots of other techniques which are More flexible and result in less artifacts. This alone mandates that you use PC to sharpen your images.
Sharpening should be the last step in the workflow, after all the levels/curves/saturation etc corrections have been done. And then too it depends upon the intended use of the picture. Web only images need different sharpening than the ones which are to be printed. Usually you shouldn't apply sharpening until it's absolutely needed. Just save your edited file in PSD or XCF format without sharpening and sharpen just before using it.

Contrast: This should be set to its minimum possible value. This results in a wider dynamic range which allows you to capture more detail. Increasing the contrast in-camera is like applying levels correction to the photo. A pixel which was 240 would now read 250 and the pixel which was 252 would be lost. Same applies to the shadow details too. So keep the contrast to a minimum and make sure that your histogram stretches from end to end. Even if the histogram is bunched up, it's always possible to strech it later in photshp but there is no way to get back the details that got lost during capturing itself.

I always check the histogram right after taking the shot. Generally I end up shooting the same thing 3-4 times before I get it right but that's ok. Good shots are worth this much trouble.

Image Size: Use the biggest size your camera can capture. I always shoot at 2048x1536, the highest my camera can go. It's all the more important if you shoot in jpeg. If you shoot at a lower resolution, the camera does not use less pixels to begin with. It can not do that. It would always capture the image at its native (highest) resolution and then resize it. And camera's resampling algorithm are no match for photoshop's bicubic resampling. So just use photoshop to resize later if you feel you don't need the full resolution but always capture the image at the highest resolution possible.

Another reason to use full resolution is if you intend to print your shots. Digital images are typically printed at 300 ppi. So if you shoot at 1024x768, you can make a print of only 3x2.5 inches. Sure you can print this upto 4x6" but it won't look as good or as crisp.

Image quality: Again, use the highest quality setting possible. JPEG is already a lossy format and further compression makes it even worse. The lower the quality (higher compression), the lesser the details in the image. That's why most of the amateur flower shots have petals made of pure color, without any detail. At a resolution of 3 megapixels, the lowest quality JPEG file out of my camera is about 100 KB and highest quality file is at 1.5 MB! Those extra pixels count for something. The quality settings are generally named as fine, superfine etc and vary from camera to camera. Make sure you know that the setting of 'High' is really higher than fine or was it the other way around??. With storage being so cheap, there is just no excuse to throw away all that information. Get a higher capacity card if you have to. I use a 512 MB card which captures about 270 shots at highest resolution.

The ultimate Winamp setup guide!

2005-07-22T14:58:00.000+05:30

When we call someone an "audiophile", we generally mean that the guy spends insane amounts of money on the audio gear. This is a wrong defintion, and a derogatory one too. To me, an audiophile is someone who is passionate about music. Who cares about music enough to invest in high quality gear. But it does not mean that we can't be audiophiles if we don't have all the high-end stuff. All it takes is right set of tools and a little bit of fiddling around.

A typical audio path looks like this: CD --> Encoder --> Winamp (Decoder) --> DSP --> Kmixer --> Soundcard --> Headphones Let's see what we can do to optimize each stage.

Encoder: If most of your music comes from internet and other sources in MP3 form, there is nothing you can do about its quality. But if you rip the CDs yourself, read on. MP3/OGG/AAc sound great but why lose quality when you can rip to a lossless format. These lossless formats are really lossless. These are like winzip for audio. You get back the exact same data, with each and every bit intact. The lossless formats do take more space (5 MB for MP3 vs 25 MB for FLAC) but with harddisk space being so cheap, there is no reason to lose quality. The best and most popular foramt for lossless audio is FLAC. I use Yahoo music engineto rip the CDs to FLAC and this plugin to play it in Winamp. Even though a few of the portable audio players support this format, it is not yet widely available on DAPs. So if your player does not support FLAC and if DAP is your primary source of music, you'd be better off with MP3. Just make sure that you use a high enough bitrate (192 kbps or greater) and LAME to encode. Some people claim that AAC has better quality at same bitrates but I can't vouch for that. These are more like Nikon vs Canon debates.

Decoder: The built-in MP3 decoder in winamp has been licensed from FhG which is pretty decent but not the best sounding one. We need something that can do dithering and noise shaping. I would use foobar but iZotope Ozone does not work with that. Fortunately, there is MAD plugin which does all this and it is free too. I must say that the difference in the decoders is very subtle and for the most part, insignificant. It is more of a purist thing. So just use whatever came with Winamp.

DSP: The next step in the chain is signal processing. This includes things like equalizer, normalizer, and other sound enhancing plugins. It takes a really good understanding of audio fundamentals to not screw up the sound with these things. You can either read tomes on this subject and try to play around with the settings yourselves. This is like using layers and masks and whatnot in photoshop to sharpen the picture where Photokit Sharpener can do a much better job at the press of a button. In other words, leave it to the pros. And the best DSP plugin for Winamp is by iZopte called Ozone. The basic plugin is free but it is so good that you'd never miss the pro version. I brings out the details in the music that you never even knew about. Especially in the mid and high frequency range. The music sounds more lus and the instrument separation improves to a remarkable degree. Listen to a well mastered CD like Gladiator or "The passion of the Christ" with and without this plug-in to fully appreciate it.

You don't need any other plugin with this, not even Winamp equalizer. I just leave the settings at default for both headphones and speakers. If you start experiencing listening fatigue on headphones, try switching to headphone preset. This mode adds a little crossfeed and HRTFs to make you feel as if you are listening to speakers. Or if you don't like too much zing in your music, try the preset with less sparkle. I found that default mode gives me the purest sound. Just play aroud with different presets and see what you like.

Kmixer: The signal coming out of DSP goes into a Windows component called kmixer. All it does is mix different audio streams so that two or more applications can play back audio at the same time. It does something else too. It resamples everything to 48 KHz (the native audio data is at 44.1 KHz) which results in a slight loss of quality. We can bypass this thing by using an output mode called 'Kernel Streaming'. It is more of a hack (experimental feature) to support applications that need very low-latency. This plugin does not support buffering (as of now) so the music might skip if you use your PC too heavily while Winamp is running. It should be fine for normal use though. In case you are not happy with KS, use directsound for Win 2K and XP and waveout for Win98/ME.

Update(Dec 07, 2005):
The updated version of kernel streaming plugin can be found in this HA thread here.

Soundcard: This is one of the most misunderstood pieces of hardware. Everyone knows that a Geforce4 is better than Geforce2 video card. You get much better frame rates and almost realistic texture. But a better soundcard? What could a better soundcard do? Play music faster?

Explaining what a good soundcard can do is like explaining color to a blind person. You have to listen to a good setup to really appreciate it. Try not to buy a Creative soundcard. These cards have been optimized for gaming with EAX effects and other 3-D trickery. You want a card built specifically for audio. Chaintech AV-710 and M-audio revolution cards are cheap and one of the best in this category. If nothing else is available, Creative Soundblaster is a pretty decent solution. Just make sure that you turn off all the so called sound enhancing settings in the soundcard driver.

Headphones: Much of what has been said about soundcards applies to headphones too. In fact, having a better headphone is more important than having a better soundcard because ear drums are a precious thing. Once damaged, they stay that way. Better buy a decent headphone and treat them nice.

A word of caution: Do not go overboard in trying to get better sound. Do not give the term audiophile its common misunderstood meaning. Listen to the music for music and become a real audiophile.

Peace!

How to simulate speakers on the headphones.

2005-06-15T12:24:00.000+05:30

One of the reasons that we have two ears (or two eyes) is that it allows us to experience this world in three dimensions. We can see just fine with one eye but we need two eyes to gauge the depth of the scene. Similarly, two ears allow us to have spatial hearing. I.E we can localize the source of the sound. We can figure out whether the sound is coming from left or right, from up or down etc.

Say we are listening to some music being played on a speaker. Typically, the sound from the speaker would reach one of the ears earlier than the other (far) ear. This is because our ears are separated by about 10 inches and sound has a finite velocity. This time difference is called Interaural Time Difference (ITD) and is generally in microseconds. And the ear closer to the speaker would receive the sound waves directly, making the signal level at this ear slightly stronger than the far ear. This difference in level is called Interaural Level Difference (ILD). Together, ITD and ILD allow us to localize the azimuth (angle in the plane) of the sound source. These are also called binaural cues since these involve both the ears. We cannot determine the elevation of the sound source using just these cues though.

The sound that comes out of the speaker and the sound that we hear are not the same. Several things affect the sound before it can be heard by us. The distance of the speakers from the ear, shape of our head, the structure of our outer and inner ear, the angle at which the sound waves strike the ear, indirect sound from reflections off the walls etc. All these influences on the sound waves are collectively called as head related transfer functions (HRTFs). Mathematically, HRTF is the impulse response of the medium that carries the sound. These HRTFs allow us to determine the elevation of the sound source. These are also called monaural cues.

But what happens when both ears receive the same level of sound at the same time and there are no external influences on the sound? Since both ITD and ILD are zero and HRTFs are absent, there is no information to localize the sound. So the sound seems to come from the center of the head. This is what happens with the headphones. Our brains loses all its localization clues and constantly tries to figure out where the sound is coming from. This results in fatigue and uneasiness, the extreme case of which happens when we listen with only one headphone (right or left). Speakers, on the other hand, sound much more natural and pleasant since brain has all the information needed to figure out the spatial location of the sound. And all the music itself is recorded in such a way that it would sound most plesant when played back on speakers only.

So if we have to simulate the speakers on the headphones, all we have to do is make ITD and ILD non-zero and introduce some HRTFs and we are set. One of the best Winamp plug-ins to do this is Speakers Simulator by Vladimir Kopjov.

Interaural level difference (ILD):
When listening to speakers, both ears receive the sound from both the speakers. But in case of headphones, left ear receives the sounds only from the left channel and right ear receives the sound only from the right channel. To simulate this on headphones, we have to take a bit of left channel and send it to right ear and vice versa. This is called 'crossfeed'. Its value varies from 0 to 80% in the above plug-in, default being 70%. The higher the crossfeed, the stronger the localization but lesser scene width.

Interaural time difference (ITD):
The sound coming from, say, left speaker would reach the near ear earlier than the far ear. To simulate this in headphones, the signal that is cross fed is also delayed by a small amount, the delay being roughly equivalent to what would be realized in practice. In the speaker simulation plug-in, the default setting for delay is 113.38 microsecond. More delay gives more scene width but less focus. If increased too much, this delay can cause some sound artifacts.

Head related transfer functions (HRTFs):
This is one of the most difficult parameters to measure because of interaction of so many different factors and different head/ear structures in different people. Lord Rayleigh modeled it assuming our head to be a perfect sphere and using wave propagation equations along a curved surface. The results show that the high frequencies get attenuated much more as they travel than the low frequencies. Though they are mathematically complex, the HRTFs are really easy to simulate. All we have to do is make the high frequencies roll off gradually and we'll get the same effect as the speakers. And unless your headphones cost $500 or more, it already has this desired frequency response (it is not so by design but rather from a desire to keep the costs down). So just make sure that the equalizer in winamp is flat for high frequencies. My Winamp equalizer looks like this.

The effect of speaker simulation plug-in is very subtle. It doesn't seem to have much effect in beginning, but after a few hours of listening, the difference is clear as day and night. And, of course, it is much more pleasant this way and you can listen for longer periods of time without having any fatigue.

Credits:

Sound Localization Using Head Related Transfer Functions
Speakers Simulator plug-in
Sound localization on Wikipedia
Comments by Chris on HydrogenAudio forums

Update:
After trying out various other plug-ins, I have finally settled on 4Front Headphones. It does not apply as much equalization as Speaker Simulator so the sound is more or less unchanged. It also gives much better stereo imaging and puts the sound directly in front of you. The default setting of 30% gives a very distorted and artificial sound but at 10%, it sounds superb.

What is a Parametric Equalizer and how to use it on Rio Karma.

2005-05-25T11:46:00.000+05:30

In it's simplest form, a speaker is just an energy conversion device. It converts electrical signals to sound waves of various frequencies. Ideally, all the frequencies in the speaker output should be at the same level of loudness. If we plot the frequency repsones curve of such an ideal speaker, with frequency (in Hertz) on x-axis and loudness (in Decibals) on y-axis, it would be a flat line at 0db. But such ideal speakers don't exist. It's really really difficult to construct such an ideal device. Even if it could be built, it would be too expensive to be practical. That means that we are stuck with ordinary speakers that don't sound so great, atleast not without some tuning.

The frequency response of an ordinary speaker looks like the outline of a mountain. I.e. the low and high frequencies are not amplified as much as the middle ones. My Sennheiser MX500 headphones have this frequency response, taken from Headroom:

The sole purpose of an equalizer is to compensate for this bias in the speakers. It brings all the frequencies to the same level of loudness. In other words, it equalizes them and hence the name "Equalizer".

Equazliers on sotware based players (Winamp, iTunes etc.) are typically 10-band and those on digital audio players are typically 5-band. A typical 10-band equalizer:

The frequency shown under each bar is called the center frequency. Just as there are no ideal speakers, there is no ideal equalizer either. I.e. instead of being a straight vertical line, each of these bars is more like a bell. And the range of frequencies falling under this bell is called a frequency band. So a 10-band equazlier can amplify/attenuate 10 separate, but overlapping, frequency bands. Because these bands extend to some distance on either side of center frequency, boosting the center frequency actually boosts this whole band, with the effect being maximum at the center frequency.

In a normal equalizer, the only thing that a user can control is the amount of amplification for each band. But in a parametric equalizer, in addition to the amplification, you can also change the center frequency and the width of each band. This additional level of control comes in handy when matching a pair of headphones to a particular device. I have set the Equalizer on my Rio Karma as:

Center frequencyuency	Gain	Width
12000 Hz	+6.0 db	4.0 octaves
2500 Hz	+0.0 db	4.0 octaves
500 Hz	-2.0 db	4.0 octaves
150 Hz	+6.0 db	4.0 octaves
40 Hz	+8.0 db	4.0 octaves

This setting makes MX500 give an almost flat response when used with Rio Karma. The nice player that the Karma is, it offers 3 custom equalizers. So I have set the other one to match my Philips HP800 cans (Based on subjective tests, since I could not find the frequency response curve for it).

To match your headphone to Rio Karma, you need the frequency response curve of Rio Karma and a way to see how your headphone and Rio karma interact at various equalizer settings. This is exactly what John M has done with his KarmaEQ application. But the headphone response curve given by headphone.com is not usable as it is. It has to be converted to a text file having frequency-gain pairs. There are many examples of this on John's site. I created a file with 30 sets of frequency-gain pairs for Sennheiser MX500. The values don't have to be highly precise. Just a simple eyeballing would do.

Now all that you have to do is import your headphone response file in the KarmaEQ and start experimenting. The idea is to have the resulting curve as flat as possible. When you are satisfied with the results in KarmaEQ, try that in Rio Karma and see how it sounds. Since sound quality is a subjective thing, the flat curve may not be up to your liking. But take that as the starting point and explore from there on. Happy EQing.

Update:
The HydrogenAudio thread on this post contains lot of interesting points on the differences between listening on headphones and listening on speakers. A highly recommended reading. Also my Rio Karma is dead :-(. I did everything I could but it just won't boot. Well..

Why iTunes worked and why Yahoo music would work too!

2005-05-12T11:27:00.000+05:30

Yahoo annoucnes it's music store.

Way back in 1998, when I started collecting digital music, I used to use Windows Explorer to organize my music and Winamp to play it. It was not the best of setups but I didn't know any better; Until i got an iBook.

The organization scheme used by iTunes completely blew me away. I actually spent 3 days cleaning up all my ID3 tags just so that I could use iTunes. It was that good. Just type a few letters and the song is there. No more mucking around with folders here and there and remembering where you put what. When iTunes for Windows was released, I made it my default player. Even though Winamp 5 has it's own media library which works similarly, it's no match for iTunes' interface.

Now, iTunes for windows is a 20 MB download but I didn't mind in the least. I just had to have it. The fact that I could buy songs through it was last thing on my mind. And that's exactly why iTunes music store is such a success and so would be Yahoo music. If you want to buy songs from any online music store, you need a special software, which is different for each store! Imagine Amazon.com and BN.com each asking you to download 20 MB big software just so that you could buy books from them! Whereas you can buy books online using just your browser, you can't buy music that way. No special software, no music. So how do you get people to download your special software? You do it the Apple way.

You make it so damn good that they start using it for their personal use, without any obligation to buy any stuff from you. The keyword here is "good". I didn't download iTunes to buy music. I downloaded it to organize the music that I already had. And that's where Real and other stores fail. Their software is designed to make it easy for you to buy music, not to organize your existing collection. You can, of course, use it for that purpose but it is torturous to use. And this is where Yahoo gets it right (as did Apple). Infact, their music player is almost a copy of iTunes but with certain new features. It can play and encode OGG and FLAC, in addition to AAC and MP3. It has a UNIX-style shell. It supports open XSPF playlist format. How much more geeky can you get? It lets you share music through Yahoo Messanger and burn CDs. It even supports iPod for non-DRM files.

Being a beta release, it is missing a few things like an Equalizer and is quite slow to load. The best thing that I personally like about Yahoo music engine is that it supports FLAC format (playing and encoding) right out of the box. Since I rip all my CDs in FLAC and iTunes does not support it, I an switching over to YME. But I'll keep Winamp around just in case and, of course, my trusty foobar2000.

Update 1:
I just ripped a few CDs using YME to FLAC format and the tags it fetched for those songs were the best I had ever seen. All the fields (song title, singer, album etc) were nicely formatted, with proper cases. I tried the ripping the same CDs in EAC and the freedb tags just made me cringe. All CAPS, wrong titles, and incomplete information. I guess YME fetches the tags from its own database instead of freedb. Well, one more reason to use YME.

Update 2:
I am really disappointed at the pace at which YME is being developed. There is still no equalizer, there are hardly any plugins, and there are some weird bugs in it. It cannot "see" an audio CD unless I login as Administrator. iTunes can read the same CDs just fine with a normal user account. It also shows Nero CD burning plugin as a removable device!

The weirdness does not end here. Turns out that YME uses Windows Media Player (or a part of it) in the background to play music. I think it is limited to just protected WMA files but I am not sure. The problem is that it even uses the graphic equalizer settings of Windows Media Player! So if you are not happy with the sound, YME help pages recommend to turn graphic equalizer off in WMP. How much more un-intuitive can you get?

Update 3:
Even though there is no built-in equalizer for YME, you can still use the equalizer on your soundcard. Some on-board soundcards (mostly found in corporate PCs) may not have one but otherwise every decent soundcard should have it. My Creative Soundblaster has a really good 10-band equalizer. And the nice thing about using soundcard EQ is that it is not limited to a particular player. It would work the same whether you use Winamp or Yahoo Music Engine.

Getting the best sound out of your PC.

2005-04-13T16:46:00.000+05:30

This article originally appeared on Osnews.com.

Chesky records, a small record label, produces what is called audiophile friendly music. To learn how to create music with this high detail and how to play it right, just read some of their articles. There is a lot of talk there on tube amps and stereo mics and horn speakers etc. Forget the high-end gear, there is one complete article on how to set up your power supply for best audio experience. But mere mortals like us listen to the music on our PCs. Though it does not even begin to compare with all that exotic gear out there, it can be set up to deliver a surprisinlgy good quality sound.

This is a highly simplified diagram of how the music flows:
mp3 file --> mp3 decoder --> processor --> soundcard --> speakers --> Ears

Each one of these stages has to be optimized if you want the best quality sound. If the mp3 file is low-bit rate to begin with, no amount of tweaking can make it sound lush and full. Or if you are using tin boxes for speakers, even a SACD would sound crappy on them. One weak link in the chain and there goes your listening experience.

Input:
MP3 technology was developed by Fraunhoffer Institute in Germany. They also provide software to encode/decode MP3 files and that is what is used in winamp by default. But these official decoders are not the best sounding ones. That honor goes to an open source (surprise! surprise!) decoder called mpg123. It supports ReplayGain and 24-bit output, both of which are lacking from the official decoder. The difference in quality may not be that apparent on ordinary songs but this plugin really shines when the MP3 file contains a lot of detail.

To use this plugin, Just copy the file 'in_mpg123.dll' to 'C:\Program Files\Winamp\Plugins' and launch winamp. Then under Preferences->input, choose mpg123 and double click on it to set its properties. Under Decoder tab, check 'Enable' and choose output format as '16 bit' (or '24 bit' if your sound card supports it). Under Replay Gain tab, check 'Enable', choose 'Album gain', and choose 'Hard Limiter'. ReplayGain will make sure that all the songs play back at the same loudness, thus freeing you from changing the volume for different songs which had been mastered at different levels. No changes are made by ReplayGain to the original MP3 file. It just puts a tag on the file which is used by the decoder to decide whether to amplify or attenuate.

An alternative MP3 decoder called MAD also has a lot of following. There is no conclusive evidence that it is better than mpg123 but arguments go on.

PS - The official mp3 encoder is also not the best one out there! That honor goes (again) to an open source encoder called LAME.

Processing:
After a mp3 file gets decoded, it is fed to a Windows component called kmixer. It does exactly what its name implies - it mixes all the audio streams being fed to it and sends them to the soundcard. This is how we can hear those annoying Windows event sounds alongwith "sweet home alabama". If this were the only thing that kmixer was doing, it won't be so bad except that the kmixer resamples everything, just to make it easier for it to mix all those audio streams. It also fiddles with signal to noise ratio and does many other undesirable things. The end result is that the audio stream comes out sounding very flat and dull. So we need to find a way to bypass this mixer thing and send the audio stream directly to the soundcard. Enter kernel streaming.

Download this plugin and just copy it to 'C:\Program Files\Winamp\Plugins'. Then choose 'kernel streaming' under Output preferences. There are no settings for this plugin (as yet). Since it bypasses the kmixer, the winamp volume control won't work with this plugin. You have to use that little speaker icon in the task bar. This is also way too loud at normal volumes, so keep the volume to a minimum when you try it for the first time. In some cases, when you double click on a file after choosing kernel streaming, the winmap might quit. This is normal. Just relaunch the winamp and it should run fine from there on.

Output:
This section applies only if you have Creative soundcard or a soundcard based on 10K2 chipset.
After a mp3 files gets decoded by winamp, it gets fed to soundcard drivers. These drivers make sense of this audio data and convert it to a format suitable for the soundcard to play. But the drivers supplied by Creative are a bit crappy and bloated. So head over to kxproject homepage and download these drivers. These drivers have a cleaner sound ouput and are much more stable than the stock ones. You also gain much more control over your soundcard settings.

Ears:
Finally, the most critical link in the sound quality chain - the ears. If you cannot tell the difference between an MP3 file and the original CD, don't despair. Most people can't. It takes time and practice and a really good audio setup to hone the listening skills. I leave you here with some commentary from an audiophile who does not just hear music, but actually listens to it. Be inspired!

Credits:
The article that made me interested in all this fiddling is here. It deals specifically with foobar2000 but the concepts are same. Infact foobar2000 comes with kernel streaming preloaded. If you can stand its UI, it is said to be the best audio player around.

In-camera sharpening

2004-12-23T07:30:00.000+05:30

When I got my digital camera, I used to shoot pretty much in auto mode. Later when I became a bit experienced and started reading photography books/articles, I started shooting manual.

I knew enough about f-stops and shutter speeds and but I didn't know anything about some of the camera settings and I couldn't find anything on the web either. Even the camera manual was of no help. Like should I increase the color saturation using settings on the camera or should I use photoshop? What about contrast? What about sharpening? How do I know whether the camera does a better job than post-processing software.

The most confusing setting among these is the sharpening. Some people say that setting the in-camera sharpening to low or normal is actually good. While some folks are dead against using any kind of in-camera sharpening at all. After searching on the internet for many days and getting nowhere, I decided to try it for myself. One fact that adds to the confusion is that the in-camera sharpening is applied before the file is converted to jpeg.

I have an olympus C-760 UZ, a variation of which, C-765, is sold in the US. It has sharpening settings of -5 to +5, in +1 increments. As part of this test, I made three exposures, one at sharpening set to -5, one at 0, and one at a setting of +5. All these pictures were taken at the highest resolution of the camera and then cropped using iPhoto. No other editing of any kind was done. The photos were taken in low light to boost the effect of the noise. Otherwise it would be very difficult to do the comparison. Here is what I found:

This first exposure was made at a sharpening setting of -5. There is little noise in the shadows, especially under the big center leaf. There is some noise in the highlights too but it is not so bad. Also notice that the part of leaf visible in the lower left corner (the pointed one) has no apparent halos around it. It actually has a little but it's hardly visible.

This exposure was made at a sharpening setting of 0. Comparing it to the exposure at -5, the noise is much more visible all across the image. Also notice the halo around the leaf in the lower left corner. It is much more prominent now. This was the setting I used to shoot at and wonder where all that noise came from. In fact I did not know that the sharpening scale on my camera went from -5 to +5. I always used to think that 0 was the lowest until I read the manual. Well, that teaches me something.

This exposure was taken at +5 sharpening and clearly is the worst of the lot, but that was kind of expected.

After finding out what in-camera sharpening did to the photos, I wanted to find out But how much worse (or better??) was this than photoshop sharpening? I tried many settings in unsharp masking and found that in most of the cases, photoshop outperformed the camera. The only time when camera sharpening was better was when the amount was too high and the threshold was too low in photoshop USM. The camera just does not provide the level of control afforded by photoshop. Another reason to avoid in-camera sharpening is that it does not allow selective sharpening.

Our world as a MMOG

2004-08-13T13:00:00.000+05:30

Have you ever felt like there is someone who controls this world, someone who punishes us when we do bad things and rewards us for doing good? Someone who keeps score; someone who gives us subtle clues when we are lost; someone who watches over us. Have you been hit by too many coincidences? Well, you are not alone.

If you have ever played The Sims, you'd be familiar with the theme. Here is an excerpt taken from Electronic Art's website -

"Your Sims now have purpose in life. Do they aspire to a life of fame, fortune, family, knowledge, or romance? It’s up to you to decide if they will be a lover or a loser, a prince or a pauper, a fool or a mastermind, and many other choices. Give them what they want and they’ll lead a long, successful existence; indulge their fears and risk ruining their lives. It's all in your hands."

Notice the last line of the excerpt - It's all in your hands!! You, the player, control what happens in your sim's world. You can make them miserable or you can make them happy. You can bring floods and famine; you can make them sick. You can make them fight and kill each other. Isn't that what's going on right now in this world, the real world? All the wars, the floods, the hunger........

People who kill someone in rage often wonder, later, what took over them. They would never have done those things in their full senses. Your mind is only as much free as the "player" allows it to be. Notice how some people don't have any purpose in life. Or take for example homeless people or drug addicts or estranged youth. They are the way they are either because their "player" has abandoned the game or he has gone for a long break.

Computer viruses are getting more and more sophisticated day by day; and so are our diseases. We have come from simple cold to AIDS today and no one knows what else is to come yet. It takes sometime for the antivirus developers to catch up with new viruses and that's why there is no cure for AIDS yet. "Those guys" haven’t yet updated their antivirus software.

In computer programming, a class is a blueprint for many objects. i.e. all the objects will have some of their properties inherited from that base class and some properties of their own. Now think of our DNA, also called "the molecules of Heredity". Most of the DNA code is identical between any two humans but there are enough differences to make each one of us unique.

There are many more things that can be explained by looking at our world as a Massively Multiplayer Online Game. We play sims, someone plays us and someone plays who is playing us. Universe inside a universe inside a universe. These ideas seem far-fetched but who knows! Whatever the "reality" is, I hope the server doesn't crash. Remember dinosaurs?

Windows + Freeware = Bliss!!

2004-08-10T22:58:00.000+05:30

Recently I upgraded my home PC and I thought, well, how about loading it with only freeware/open source software (On top of Windows XP)? So I started looking around and found some excellent freeware, alongwith a lot of trash. It took some work to evaluate all the applications but in the end, it was worth it. I have nothing against Micro$oft or any other ISV making money though. It was just that I was on a tight budget and I wanted to see whether it was really possible to live without commercial software. Here is what I have settled on in the end.

Audio:
Nothing beats good ol' Winamp (classic) here for playing songs. For cataloging my music collection, I use MPEG Audio Collection. It is very lightweight and can read all types of audio files. It uses Windows default player (winamp here) to play music. Apple's iTunes is also good but it's very resource intensive, especially if you choose to normalize the volume. I use both MAC and iTunes as they serve two different purposes. With MPEG Audio Collection, you can catalog all your CDs and search for the songs without needing the original CD. On the other hand, iTunes does not allow you to play songs based on the folders (location) but you can do that in MPEG Audio Collection.

What good is winamp without some music for it to play and again I turned to freeware. There are more sources of legal free music on the internet than we know or care about and that's where irate comes in. When you run this program for the first time, it will download 10 tracks at random from different sources. You listen to each song and rate it on a scale of 0 to 10. Say you rate country songs higher than Blues. Then the program will start downloading more of country songs. With a little bit of training, the program can give you a hit ratio of 70% or more. And these are not streams or samples or pirated songs. These are free-as-in-beer full songs. And you can just right click on an artist's name and visit their website.

I use CDex for CD ripping and mp3Trim for occasional mp3 editing.

Video:
Even though I got a copy of PowerDVD when I bought my DVD-ROM drive, I don't use it as I am planning to use only freeware. But this is one area where I was disappointed. I could not find any good DVD players which are also freeware. The best I could find was VideoLan. Its interface is not as eye-candy as PowerDVD but it does play every imaginable file format, even those files that other softwares refuse to play. It's very lightweight and highly customizable. Don't be put off by its interface.

Updated on Aug 20, 2004 - BSPlayer is a terrific piece of software which specialises in video and divx playback. It has ton of features and plays all kind of media files.

For video editing and capturing, the most popular freeware application is VirtualDub. For casual use, Windows Movie Maker 2 is a good choice.

CD/DVD writing:
I had always used Nero for this but CD Burner XP Pro is an excellent piece of software. It's hard to believe that something like this is free! It works with IDE, USB, IEEE1384, SCSI CD/DVD recorders, can create bootable discs, iso files, audio CDs etc. And best of all, it has a very powerful CD ripper, featuring ogg and wma, in addition to mp3. It has made me forget Nero and that's saying a lot.

Free version of DeepBurner is also an excellent software. Though pro version has much more functionality, free version is more than enough for daily use.

For viewing image file, I use CDmage. It supports more than 25 formats, including .nrg, .ccd, .iso, and does an amazing variety of things.

Productivity:
The first application that comes to mind in this category is OpenOffice. It should serve you well for almost all of your needs. In case you get a complex layout file that was created in MS Office and OpenOffice is unable to render it properly, you can always download the free viewers for Word/Excel/Powerpoint/Visio from Microsoft site. I tried Abiword also but I did not like its interface that much. And since you'd be installing OpenOffice anyway, there is no need for any other word processor.

I use Keynote for note taking. It is very lightweight and has a nice tabbed interface. You can also use free version of Treepad but it does not support images.

Internet Browsing:
There are two choices for browsing the internet. Firefox and MyIE2. I do not prefer firefox that much because of its memory leak problems. Use it for 10 minutes and it would start taking 60-70 MB of RAM. But it's very secure and comes with a built-in pop up blocker. It is recommended only if you have atleast 256 MB of RAM. Otherwise it's better to use MyIE2 which is a shell around IE. It also supports tabbed browsing and has many more features than firefox.

Mail:
Thunderbird from mozilla organization. Though I mostly use gmail/yahoo for personal emails (at home).

IM:
Gaim is a multi-protocol IM software, supporting Yahoo, MSN messanger, AIM, ICQ, IRC, and few more. It allows you to compose your messages using its built-in WYSIWYG interface. And if your friends have multiple accounts, it can club them into one single entry which you can expand if needed. It even has RSA encryption (through a plugin). Miranda is another very good IM software.

Anti-virus:
I use a product (appropriately) called antivir from H+BEDV. It is very light on the resources and gets updated daily. You can also use AVG anti-virus from Grisoft but antivir gives you much more fine-grained control over dealing with viruses.

Firewall:
I use Kerio personal firewall from Kerio technologies. It's highly customizable and in addition to providing network security, it also provides very good system level security. Meaning it will alert you if an executable tries to launch another executable. And you can create custom access rules for different programs. Zonelabs also gives a free version of their famous zonealarm firewall but free version does not allow you to specify policies for individual programs. Though Sygate personal firewall is also good, you can create only 20 custom rules in free version. Needless to say, do not trust Windows firewall that comes with XP SP2.

Data Compression:
7-Zip is the best freeware compression tool out there. Its proprietary compression format 7z claims 30-70% better compression ratios when compared to regular zip format. It even supports rpms and deb formats,in addition to all the other major ones.

Imaging:
irfanview is the best freeware image viewer/editor that I have used till date. It supports any image file format that you have ever heard of. And its editing capablities are more than sufficient for casual users. You can crop/convert/apply filters/get EXIF/create slideshows/change color depth and do many more things. Don't forget to install the plugins though.

For everything else, there is of course gimp. One more good application is Paint.net. It is under active development but looks very promising. You need .Net runtime for this though.

File Utilities:
SC-DiskInfo will quickly show you how much disk space is being used by different directories. Run it on 'Documents and Settings' folder and be surprised by the amount of space being taken by temp files. It displays the space usage in a very nice bar graph. If you keep wondering where your disk space has gone, get this software.

To recover files that got emptied from recycle bin also, Restoration is a very handy utility. It woks on all versions of Windows and does not need any installation. Just unzip and run it, preferrably from a floppy.

Misc:

Some other programs that I use are - Spybot Search&Destroy for removing spyware, CmdHere power toy from Microsoft (for Windows XP), free version of Joel Spolsky's CityDesk for (clean) HTML composing, and Clip Path for capturing the full path of a file/directory to clipboard.

There are many more programs out there than I could ever test. But these should be sufficient for an average home user. And if you like any of these, consider donating some money to the developers. Even five or ten dollars go a long way towards paying bandwidth and hardware costs. You can also help with documentation or programming. At the minimum, send an appreciation email to the developers and thank them. It does wonders for their motivation and encourages them to continue working on the application. Some sites like SourceForge sell T-shirts and you can show your support by buying them. Remember, no help is small help.

Let's go exploring:
If you want all this software and more on one CD, here are some links to explore -
The Open CD - Offers CDs and free iso images containing most of the applications listed here and many more. Updated regularly.
GNUWin - This is similar to OpenCD but offers a lot more software. It was lagging a bit in terms of updates (at the time of this writing) by abount two months.
Open Source Software CD - Offers a lot of software and is updated regularly. Better than both 'The Open CD' and 'GNUWin'.

Dollar Cost Averaging

2004-08-06T08:45:00.000+05:30

The basic principle that stock markets work on is “buy low, sell high". This works fine if you keep a close eye on the market and time your buying/selling accordingly. but this may not work in all the cases. Some people stop buying when the price starts rising, fearing that they might overpay. Others will start selling when the prices starts coming down, afraid that they might even lose whatever little profit they would be making. When you start timing the market, you might miss its best performing cycles.

A way to stop timing the market and take the guess work out of investing is DCA. It basically involves investing a fixed amount of money at fixed intervals of time, regardless of how the market is doing; Say investing a total of $12,000 over a year, $1000 per month. Since the amount of money being invested is fixed, you’ll be buying more shares when the price is low and less shares when the price is high. So the share prices will average out in the long run and you might end up making a tidy profit.

There are a few things worth noting about DCA. First one is that it does not always guarantee a profit. You make profit in DCA only if the ending price of the stock is higher than the average monthly price. And that cannot always be guaranteed. Studies have shown that DCA under-performed 2/3 of the times when compared to lump-sum investments. Second thing to note about DCA is that you have to have a lump-sum amount to start DCA. If you invest, say $1000 every month, out of your salary, then that is not dollar cost averaging. DCA comes into picture when you consciously decide to invest regulary as opposed to doing a lump-sum investment.

DCA is also known as ‘Contant-dollar Plan’.

A Comparison of PDS and PDSE

2004-06-18T12:37:00.000+05:30

PDSEs (Partitioned dataset extended) were first introduced by IBM in MVS/AFP 3.2 in the year 1989. Even though regular PDSes were quite adequate for normal tasks, many IBM customers were not happy with them. One of the IBM user groups, SHARE, did a project on MVS Storage Management and published a white paper. This paper summarized the findings of the project and asked for a number of improvements and new features to then current PDSes. Another IBM user group, GUIDE, also published their requirements and asked for similar changes. IBM listened and the result was PDSE. But first a bit of background on plain old PDS to understand its limitations.

A PDS is essentially made up of two parts - a directory and a few members. The directory is a set of contiguous 256-byte blocks, present at the beginning of the dataset. Each of these directory blocks contains a 2-byte count field at the beginning and from 3 to 21 directory entries after that. There is one directory entry for each member in the PDS. Each directory entry contains - 8 byte member name (padded with spaces, if needed), starting position of the member in the PDS (in TTR addressing format) and some optional (up to 62 bytes) user data.

A directory block will contain only as many complete entries as can fit in 254 bytes (2 bytes are reserved for count field). The remaining bytes are left unused. The length of the user data determines how many complete entries can fit in one directory block. The 2-byte count field contains the number of used (also called 'active') bytes, including the bytes used for the count field.

This directory structure was the reason why there was a need to improve PDS. When IBM introduced PDSE, it replaced the rigid directory structure of the PDS with a new flexible scheme and also brought in many new features. And all this was done while keeping the PDSE backward compatible with PDS. That means that except for very low-level (hardware dependent) processing, users need not even be aware of what they are dealing with.

Some of the new features introduced in PDSE and their comparison with PDS is given below -

Expandable directory size

The number of directory blocks in a PDS is specified at the time of its creation and can not be changed after that. Also the space for all the directory blocks is allocated at the time of creating the dataset. Lets say that a PDS was allocated with a directory block count of 20. Assume that an average 256-byte directory block holds 10 directory entries. So now this PDS can contain at most 20x10 = 200 members. But what if you use this up and want to create 201st member? Tough luck!

PDSE solved this problem by creating an indexed directory structure. Now each directory entry points to the one coming next to it. This matters because now there is no need to allocate all the directory blocks at the time of creating the dataset. This also means that they need not be contiguous and need not be fixed in number. They can be interleaved with the member data blocks and they indeed are! When you want to create new members, a new directory block is created in the next available storage and the pointers updated.

Note that its only the directory blocks that increase in number. The total size of the PDSE does not grow beyond one primary extent and 123 secondary extents. In other words, the directory can expand only if there is enough space in the dataset. The maximum size of the PDSE itself remains fixed.
Better search and insertion

The directory entries in a PDS are stored in the alphabetical order of member names. So if a new entry is to be created, all the entries coming after it need to be shifted to make room for it. This is called 'Ripple Stow' and it results in many I/O operations, making the whole process a lot slower. Same holds true for searching for a member within a PDS. The entire directory needs to be scanned to locate a particular member.

Since the directory in a PDSE is an indexed structure, there are no such performance problems in PDSE. So it always takes the same amount of time to search/insert a new member whether it starts with 'A' or 'Z'.
Improved sharing facilities

The locking mechanism in a PDS operates at the dataset level. If you want to update a single member in a PDS, you need the exclusive access to the entire dataset. No other user or job can update any other member in that PDS during that time. While in a PDSE, the access control is implemented at the member level. So two users can update two different members at the same time. Makes you wonder how people worked before PDSE came.
Better use of disk space

When a PDS member gets deleted, the space that gets freed up is not used for allocating new members. Since the deletion of a member causes the deletion of that directory entry, the pointer to that member location is lost and so is that space. As the members get allocated/deleted during the lifetime of a PDS, the amount of this wasted space keeps growing. This wasted space, also called PDS gas, can be as much as 40% of the total allocated space. So the PDS needs to be compressed periodically to re-claim this space. The compression can be done by either typing 'Z' in front of the PDS name (in ISPF) or by using IEBCOPY utility.

On the other hand, a PDSE keeps on re-claiming the freed space automatically, using a first-fit algorithm. Issuing a 'Z' command or doing an IEBCOPY has no effect on a PDSE.

Also, whenever a new member is created in a PDS, the data blocks allocated for it have to be contiguous. But there is no such restriction in a PDSE. So the space re-claimed from deleted members can be allocated to new or existing members. This results in a much better space utilization.
Improved dataset integrity

If a PDS is opened for output in a sequential mode, e.g. if an IEBGENER step omits the member name and uses only the PDS name, say in

//SYSUT1 DD DSN=Some.input.sequential.file,DISP=SHR
//SYSUT2 DD DSN=PDS
the entire directory would get destroyed all the members would be lost. If a similar thing is attempted on a PDSE, The job would terminate with a abend code of S213-4C and the PDSE would remain intact.
S213-4C : WHEN OPENING A PDSE DSORG=PS WAS SPECIFIED, BUT NO MEMBER WAS SPECIFIED.
Hardware independence

PDS uses an addressing scheme called TTR (Track-Track-Record) which is based on the DASD geometry. TTR addresses are stored in hexadecimal format. So an address of X’002E26’ would mean track number X'002E' and record X'26'. The name TTR comes from the fact that first two bytes of the address denote track number and third byte denotes record number. This dependence on the DASD geometry makes it very difficult to migrate PDS from one type of DASD to another one, e.g. from 3380 to 3390.

The PDSE addressing scheme is not dependent on the physical device geometry. It uses a "simulated" 3-byte TTR address to locate the members and the records which makes the migration easier. Incidentally, this simulation of addresses places some limitations on the number of members and the number of records per member in a PDSE. A TTR address of X'000001' in a PDSE points to the directory. The addresses from X'000002' to X'07FFFF' point to the first record of each member, which is why there is a limit of 524,286 members. The addresses from X'100001' to X'FFFFFF' point to records within each member, which is why there is a limit of 15,728,639 records in each member.

Verifying a VSAM file

2004-06-04T08:50:00.000+05:30

When a VSAM file is opened by a job in the output mode, a flag in the VSAM catalog called "open-for-output" gets set to 'ON'. This flag does not get turned off until the job ends successfully i.e. until the job closes the file normally. Or in case we are editing the file manually in file-aid, the flag gets set when we open the file and gets turned off when we close it. But say the job goes down halfway through the update process or our TSO session expires before we could close the file. The "open-for-output" flag remains turned on. What happens to the file now?

When next time a job/user tries to open the file for output, the file manager trots off to the catalog to turn on the flag. But the flag is already on!! So it guesses (somewhat optimistically) that some other job might have opened the file. It issues a GRS (Global resource serialization) enqueueing on the file to find that out. If the file is not open anywhere else, it comes to know that it was not closed properly during its last use and its time for some catalog cleanup. Enter VERIFY.

Verify is a record management macro like get or put. In our case, the open processing issues an implicit verify against the file. This can be confirmed by the IEC161I type warning messages (RC of 56 and 62) in the sysout. The verify macro will compare the ICF catalog information with the physical VSAM cluster. It starts reading the VSAM dataset CI by CI, starting with the High Used RBA. It compares HURBA value, number of index levels, system time-stamp and many other fields. If the verify is not successful, it will issue a warning message with a return code of 116 (X'74'). Two things worth noting about implicit verify macro:

It will not correct the catalog information. That is the job reserved for IDCAMS verify. It will just issue some warning messages. It is up to us to figure out what to do next. We can continue processing but the data integrity won't be guaranteed.
Implicit verify is not issued if the file is being opened for input or reset processing. Or if the VSAM file is of type LDS.

IDCAMS verify is an explicit verify command. When an IDCAMS verify is issued against a file, it opens the file, calls the record management verify macro and then closes the file. And it is at the time of closing the file that the ICF catalog gets updated and the "open-for-output" flag gets reset. Two things worth noting about explicit verify macro:

A successful verify does not guarantee that everything is fine now. The catalog statistics may be invalid; the file might have duplicate or missing records. HURBA may be off by a few bytes.
IDCAMS verify does not update the catalog or the VSAM control blocks directly. It relies on the implicit verify and VSAM close processing to do its job.

So whether the verify is successful or not, the structural integrity of a VSAM file is not guaranteed. Its always a good idea to take standard recovery actions on such files.

Reading with a Purpose!!!

2004-04-20T12:31:00.000+05:30

"Some books are to be tasted, others to be swallowed, and some few to be chewed and digested".

- Francis Bacon, sixteenth-century English philosopher

Turning the pages of a magazine while sitting in a dentist's clinic is very different from pouring over the details of your new web-based transaction system in your office. Even though what you are doing in both the cases is going through some text or images, there is a big difference in "why" are you doing so. Or in other words, your purpose of reading is not the same when you read different things. You read a magazine in a dentist's clinic to kill time. But you read a technical specifications document to really understand how the damn application works. And herein lies the secret to effective reading.

Let's get back to reading a magazine in a dentist's clinic. The moment you pick up the magazine, you know in the back of your mind that this is just for killing time. You know that none of this stuff is worth remembering. And when your turn comes, you just drop the magazine and go off. You had a clear purpose before your started reading that magazine. Now the trick is to apply the same rule to all the reading that you do.

So, before you start reading anything, ask yourself -

Why I am reading it? Is it for killing time? Is it for really learning something?
How will I be using this stuff? Will it make me do things differently? Will it make me plan better? Will it get me a girlfriend?
How important is this stuff to me? Is it worth remembering? If so, for how long? Is it just good for gossiping or has it got some real value?
What level of detail I want? Do I want just the big picture? Can I read just a section to get what I want? Or should I read the whole book?

It takes less than a minute to establish a purpose but it can save you hundreds of hours. Hours that were spent reading 'Boing Boing', just because you felt you had to read that. Reading without having a purpose is like traveling without knowing where you are going. It might seem like fun but it won't get you anywhere. Add to that the tools at our disposal to flood us with information and you have the ultimate time-wasting machine.

Every time you read something, think of its purpose. Note it down. Then think of what made you start reading it in the first place. Compare the two. You're guaranteed to be surprised.

So next time you start reading a popular blog, ask yourself!!

Pragmatic Programmer

2004-03-24T11:18:00.000+05:30

Started reading the book 'The Pragmatic Programmer : From Journeyman to Master'. This is one of the best books out there on the subject of Software Engineering. Absolutely amazing. It's one of those books that make you say "I wish I had read this earlier". Very light reading. Doesn't use any of the obscure jargon found in other books. A must read for every programmer.

EXCPs

2004-03-19T13:59:00.000+05:30

One of my teammates remarked that the Exception count for his job was not increasing and that it might be in a loop. He was monitoring the job in SDSF and apparently there was no visible activity. The discussion that we had on this is the reason for this post.

EXCP stands for EXecute Channel Program. It does not stand for "Exception". These are the I/O subsystem hardware driver programs that do the actual data transfer between the DASD (Hard disk) and the system memory(core/RAM). Channels in mainframes are similar to Buses in the PCs. They are basically the electrical paths to carry data. So each time a trip is made to the DASD to fetch data, EXCP count goes up by one. This is the reason we say a job is looping if the EXCP count is not increasing. Note that the count goes up by one for each block of data transferred. So a data transfer of a single 4K block and a single 32K block will count for one EXCP each.

There is a little caveat here. If the data being fetched is in a DB2 table, the EXCP count will NOT go up even though large amounts of data are being transferred. That is because the EXCPs are logged by a part of MVS called SMF (Systems Management Facility) while DB2 I/Os are handled by MMF (Media Manager Facility). So those I/Os don't show up on the EXCP reports.

Mandrake Linux 10 Preview

2004-01-06T18:38:00.000+05:30

This article originally appeared on Osnews.com.

Mandrake Linux Preview Edition pretty much defines the shape of things to come in Linux land in 2004. With Kernel 2.6, KDE 3.2 beta and XFree86 4.4 beta, it doesn't leave much to be desired. This article refers to cooker snapshot as of December 31, 2003. Please note that this release is not a beta release. This is not even an alpha release. Its just something put together to show what we can expect from Mandrake 10.0. This release comes on only two CDs so some of the packages are missing. And as there are bound to be lot of bugs in this kind of release, I'll be concentrating more on the usability aspect. So let's see if it is worth drooling over.

Install:
I did not want burn the iso images to the CDs so I chose to install directly from the harddisk. First thing I did was to bust the iso files using winrar. Then inserted a floppy and double clicked on 'rawwritewin.exe' in directory 'dosutils'. Pointed it to the directory called 'image' and chose 'hdcdrom_usb.img' . The boot floppy was ready in 4 minutes. All this was done from within Windows. Booted using the boot floppy just created. Chose 'harddisk install' method and pointed the installer to the place where the packages were lying. Note that the installer expects names like hda5 while asking for the package location. It won't understand what C:\ means. and while busting second iso, make sure that the rpms from this image are extracted in a folder called RPMS2 under 'Mandrake' directory. Otherwise the installer won't be able to find them. The install process itself is essentially same as before. I chose Hindi as one additional language and the installer offered to install 'Devanagiri' keyboard layout. Very helpful. While installing individual packages, the installer does not show the package version number. Not a big deal though but I am used to it from my Redhat days.

I had chosen ext3 as the filesystem and the whole install process took about half an hour. No third party ads were displayed during the install. Even though there is a folder called 'advertising'', it just contains Mandrake's own promotional ads.

First Boot:
One of the first thing that hits you when you login is the responsiveness of the system. The system seems really fast. Even though lot of RAM (640 MB DDR) and new Kernel/KDE/XFree86 helps but it certainly is much much faster than Redhat 9 and Windows XP on the same hardware. Sometime I got the creepy feeling that the system was anticipating my mouse movements and bringing up the screens even before I could click! The directory listing of same shared drives (songs etc) was coming up much faster than it was in Windows XP. and these shared drives are FAT32, something Windows is supposed to specialize in. I wonder how would it be when pre-linking is enabled. Hope they add that to the final release.

I have two harddisks, first one containing just the OSes and the second one containing songs, docs, videos etc. The second harddisk has four partitions with volume labels songs, docs, videos and junk. Not only the system automatically mounted all of them under /mnt but get this, it created all the mount points by reading the volume labels of the partitions! I was awesome. No other distro has ever done that. Infact I keep a copy of fstab in a separate partition which I use after installing a new distro. Guess I won't be needing that anymore. Though it mounted my USB harddisk also by itself, it did not read the volume information from that and instead named it as win_c3.

I had created one normal user account 'manish' during install. When I booted for the first time, the system automatically logged me in as 'manish'. But I wanted to login as root to do some onetime settings. I thought I'll just logout and maybe I'll get to see the login prompt. On logging out, it showed the login prompt but there was no place where I could enter username. I had to click on the name 'manish' but this time it did ask for the password. I Changed some settngs in the login manager and made it show 'root' on the login prompt. I know its not a good idea to display 'root' as one of the users, atleast a text field should have been supplied where I could type username 'root'.

Package install:
It turned out that the package 'Wine' got left out during the install. Actually I don't remember seeing it anywhere during the package selection. Anyway, I launched rpmdrake but it ended up in dependency hell. Launched urpmi and gave 'urpmi.addmedia local' and gave it the path for 'Mandrake' directory where the rpm packages were stored. It could not load the rpm package list. Gave an error message saying that the files hdlist and synthesis.hdlist could not be parsed/located even though they were peresent. Mucked around a bit more with urpmi but could not make it access local rpm files. Then stumbled upon the GUI tool called 'Software Media Manager'. The same thing that I was trying to do by command line worked flawlessly in GUI. The local rpm files list got created and finally I was able to install 'Wine'. Well, all I can say is I am yet to find software install nirvana. And I also wonder if it is so tough to put every object file needed in the same rpm package.

Multimedia:
This is the first Mandrake release that has got a beep out of my onboard AC97 audio controller. I had to fiddle with audio mixer a bit but it worked in the end. Clicking on a mp3 file brought up totem player. I was hoping to see xmms but nothing a few mouse clicks can't fix. I guess the default should have been xmms in the first place. Xmms here is highly unstable though. It froze up on the first mp3 itself and took the entire system down. But I guess thats ok in this kind of release. And it still does not contain extra skins and equalizer presets.

There are only two media players included, Mplayer and Totem. Xine is not present and neither is libdvdcss even though there is a package present called xine-plugins. Wonder what that does. Video files play by default in totem. I changed the settings to make Mplayer the default for such files. It showed a progress bar saying 'changing system settings'. Clicking on a video file now brought up totem again. Also I had associated dat files (VCD clippings) with Mplayer but it had no effect. It still brings up the dialog box asking me to choose a program. These seem to the problems with KDE rather than with the distro but since KDE itself is beta, you can never be sure.

There are lot of programs installed to deal with image files. Infact, there are too many of them, may be 8 or 9. And all of them do more or less the same thing. Though it is good to have choice, this just seems like overkill to me. Good old Gimp is also present but it is quite old 1.2.5 version. With this kind of release, they could have included 1.3 beta and nobody would have complained. It contains much better menu layout and CYMK support.

Productivity:
OO.org 1.1 is present along with KOffice 1.3 beta. Loading time of OO.org has improved a lot since 1.0 but it is still not fast enough. And I think whatever speed gain I saw was because of the new kernel and new XFree86 etc. Filters have also improved for MS Office documents but a lot remains to be done. I opened a simple word document with a few bullet points and all the bullets (in this case, small round dark circles) had big square grey boxes around them. It looked plain ugly. It can ofcourse be fixed but defaults should make sense. KOffice is still very buggy. KWord froze up 2-3 times on opening the same file and just won't get refreshed. Personally, I think these guys should merge with OO.org. There are 7 text editors present, one for each day of the week I guess. Incidentally, I am typing this review in KWrite while playing around with this release.

The menu layout is pretty intuitive for office applications atleast. Instead of grouping them by brand, they are now grouped by functionality. So all the word processors go under 'Wordprocessors'. GNOME dictionary turned out to be very helpful in checking some word meanings but it needs internet connection. It would be much better if there is an offline dictionary included, somethink like Wordweb for Windows.

Internet:
Plenty of stuff here. Galeon is also present in addition to Mozilla and Konqueror. And Mozilla still retains its ugly classic theme as default. This point has been talked about so much in online communities but nothing seems to convince the package developers to change it. Flash plugin is not present and neither is Java. What is the point in putting ton of new features in each version if it can't do the basic stuff right? Konqueror was horrible at reproducing the fonts as intended by the web page. I went to www.osnews.com and the page looked terrible in Konqueror. But the same web page looks gorgeous in Mozilla. Maybe there are some font settings that can be changed but default in Konqueror is just hit or miss.

I used gaim to connect to Yahoo chat server and everything worked right the first time. There is an application called 'Screem' to build web sites. Its something similar to Yahoo site builder though not that powerful or that intuitive. I could not find site templates which is the first thing it should have had.

In terms of internet security, the system has a firewall called Shorewall. I chose the standard level of security accepting the default settings. The Zone alarm firewall test on Windows XP shows all the ports to be in stealth mode. Means that it eats up all the incoming ICMP packets and it appears that there is no PC at this IP address. I decided to see how does Shorewall fare. Went to the site 'http://scan.sygate.com' and gave a port scan. It showed all the ports to be in 'closed' state only. That means that someone could still see that there is a PC at this IP address. It is secure but I won't have worried had it been in 'stealth' mode. I then changed the security level to 'paranoid' and sure enough, the port were shown to be 'Blocked' which means they are in stealth mode. I felt better but now I could not access my shared drives mounted under /mnt. Oh well..

Did a Nmap scan also on the PC and it showed only port 6000 to be open which was being used by X11.

General usability:
One of my pet peeves is the default application bindings in Linux distros for the common type of files and Mandrake doesn't fare any better than others. eg double-clicking on an iso image file brings up an application selection dialog. Now the most common use of an iso file is to burn it on to a CD and K3b should have been configured to do so by default. Xandros does this right. Another example is .dat files. There are used in VCDs. Now the only thing I can do with a dat file is to view it. So Mplayer should have got fired up and played the movie. Since this is just a preview release, maybe the things will change in future. Moreover with all the distros having more or less the same standard set of packages, these are the only areas where a distro can differentiate itself from others. I am sure we'll be seeing a lot of usability enhancements in 2004.

The system seems stable enough for daily use. The only two things that were acting up were xmms and KOffice. Everything else seems to be working fine. I would like to say here that the speed of the system reduced a bit after using it for 2-3 days. Maybe because of all those log files getting written.

Conclusion:
This is going to be a big release for Mandrake especially considering their financial situation. It won't be wrong to say that this is the release that can make them or break them. Hope they get this one right. Enough has been written about KDE 3.2 beta and how it still needs a lot of polish. As for me, I'll be giving Fedora core 2 a spin and then decide for myself. If Fedora offers same levels of performance, I don't mind installing a few multimedia packages and getting on with my work.

Introducing Lorma Linux 4.0

2003-12-24T14:29:00.000+05:30

This article originally appeared on Osnews.com.

Lorma linux 4.0 is the first distribution to be based on Fedora Core, outside of Redhat, that is. It is one of those new breed of single-CD distros that try to include only the best-of-breed applications.

It is primarily a desktop distro and does not include any of the software for setting up http/ftp/mail and other kind of servers. Though the matter of choice vs simplicity is a debatable issue, if you want 5 text editors in addition to OO.org and Koffice, this distro is not for you.

Lorma linux tries to offer what Fedora left out in its release. Redhat users are all too familiar with the process of installing lots of packages after installing the main OS. Its kinda like what you do on Windows only on a smaller scale. But not so with Lorma. Here you get everything that you wish Fedora had included. Don’t get me wrong on this one. I respect the stand taken by Redhat concerning legal issues surrounding mp3 plug-ins and DVD content (un)scrambling systems and other things. But looking at it from an end-user’s perspective, it’s a bit of inconvenience. And that’s exactly what Lorma Linux promises to relieve us from.

I have been using Redhat linux since version 7.1 and my main OS for day-to-day work is Redhat linux 9. I have not used Fedora Core 1 and I don’t plan to do so either. I would definitely be using Fedora Core 2 though. That said; let’s see if how does Lorma linux fare in comparison to Redhat Linux.

Install: The install process is identical to that of standard Redhat linux, except one thing. It gives you a choice of 5 package groups, what it calls “Installation Classes’. Different pre-defined package groups will be installed based on what installation class you choose. This approach has its own good and bad points. It’s very good for people who are new to Linux as it saves them the trouble of choosing from thousands of packages. But on the other hand, ‘office workstation’ installation class does not include development tools and the ‘classroom workstation’ installation class includes games!! It should be noted that the users can either accept these pre-defined package groups as it is or they can make changes as they wish by selecting/deselecting packages. Very handy feature. Anyway, the good old custom mode is present too. As any regular reader of OSnews would have guessed that’s what I chose. Install went smoothly, except for one small hiccup which I describe in the support section.

Multimedia: This is one of the biggest selling points of this distro. The phrase doesn’t make much sense when you consider that this distro is free but anyway. It comes pre-installed with mp3 plug-in for xmms. Though why they don’t include equalizer presets is beyond me. In fact, none of the distros that I have seen so far include these. It’s a very simple thing to import the presets from winamp and I wish this distro had done so. Maybe in the next version! Mplayer comes with divX and OpenDivX plug-ins pre-installed to watch DVDs ripped in divX format. Among CD writers, k3b has been included which I believe is the best CD burning application in the Linux land.

Internet: Mozilla comes with the Modern theme as the default, which looks beautiful compared to that crappy Classic theme and it comes with Flash plug-in pre-installed. These are small things but these are what give the users a better experience. Version 3.1 of Lorma Linux had java pre-installed but it was removed in release 4.0 because of space constraints. I wish they had retained it and removed some other application instead. One candidate for removal could have been Scribus, used for desktop publishing. It is a very specialized application and there are very few people who would be using it in production environment.

Lorma Linux comes with Yahoo messenger pre-loaded for instant messaging. With recent issues regarding gaim and MSN chat rooms access, who knows if Yahoo would also block access to gaim users. I find it very reassuring to have Yahoo messenger ported to Linux as all my friends are not Linux users and most of them use Yahoo messenger anyway. For enjoying streaming audio and video, RealPlayer is present. It is the freeware version and it forces you to register it when you launch it for the first time. I wonder why it does that. I just gave it a dummy mail-id and off I went.

Support: Lorma Linux is a project of an educational institution in Philippines. It is not a commercial distro in the true sense. So the basic source of support is the user forums. While I was installing it using VMware, the installer was hanging after installing a few packages. I tried to install it 2 more times but each time it would hang on one package or the other. I posted my problem in the user forum and I had the answer within 5 minutes (I guess it was from one of the developers). Turned out that VMware was using SCSI emulation for the virtual harddisk by default. I changed it to IDE as suggested in the answer to my problem and the install went smoothly. This kind of response is very rare in the freeware world. And I should mention that even I didn’t know at the time of posting the question that I would be doing a review of this distro. So there is no question of favorable treatment or any such thing.

Office/productivity: All too familiar OOffice.org is present, version 1.1. Keeping in with the philosophy of single-CD distros, KOffice has not been included, neither is abiWord.

Misc: Wine package which was removed in Redhat 9 because of developer constraints is present in Lorma Linux. Though you should be careful while selecting packages if you want to install it. It is present under ‘gaming packages’ and it is the only thing that is present there. As gaming is not the only thing it is used for, it should be present either under ‘System utilities’ or ‘Miscellaneous’. Making it a part of the base package would also be a good thing.

Cons: There are lot of places where the installer still uses the word ‘Redhat’. For example, while installing, it gives you a message saying ‘Welcome to Redhat Linux’. There are many more places where it refers to itself as Redhat Linux. Not sure what would be Redhat’s stand on such issues but it should be changed as soon as possible. Synaptic is not present though there is an entry in the start menu. Nothing happens when you click on that. Guess the it got left out during packaging.

Conclusion: This distro has a great potential to become a mainstream player only if it gets its target audience well defined. Single-CD distros just don’t have enough room to please everyone. With Redhat itself going after corporate users, Lorma Linux should concentrate more on home users. That means installation classes like ‘office workstation’ have to go. Second thing is that as of now, it is not doing any value addition other than supplying packages missing in Fedora. It would be good if it did something like what Lycoris does. Install the whole CD as an image. With just one CD, the package selection should not be an issue.

Considering the nature of its origin, Lorma Linux people have done a very good job. It can become a serious contender for desktop by taking care of a few things. With Redhat’s polish and Lorma’s package selection, it just can’t go wrong.