CTO Tech Blog

Wouldn’t it be great if all unsolicited commercial email, or spam as it’s more commonly known, came with a tag in the subject line that identified it as spam?  Not surprisingly, the people who send spam don’t see the value in that idea.  The job of defending networks and users against spam is on the shoulders of network administrators and users.  Understanding what defences are available and how they work will help users and network administrators choose the best solution for their environment.

Spam filtering can be done in any of four points in the path of an email; as it is sent, before it reaches the destination server, on the destination server, or at the end client.  Checking for spam as it is sent seems, on the outside, to be futile since someone intending to broadcast spam would certainly not do it.  In fact, a large portion of spam is sent using malware that is unknowingly installed on end user hosts.  Checking for spam leaving a host would identify that there is a problem which could then be corrected using malware tools.  Companies that host their own mail servers, as well as many hosted servers check email for spam as it arrives or while it is being processed by the mail server.  Filtering spam before it arrives at the server reduces network traffic and isolates the server from malware that may be contained in spam, but often limits the ability for users to allow email that may appear to be spam from people or businesses that are known.  This is called white listing a sender or email domain.  Moving the spam filter to the mail server often adds simpler configuration tools to allow administrators or users to adjust the white list (desired) and blacklist (malicious) settings either on a global or per user basis.  Client based filtering allows users to individually decide how to identify spam and how to act on each message based on the rules that they configure.  Any or all of these types of filtering can be used depending on how much control a network administrator wants to retain or offload to users and most solutions are a hybrid mix.

Regardless of what stage on the email flow a filter is testing for spam, the same criteria can be used.  Most spam filters use a number of tests in a certain order with specific settings for action and logging for each test.  The best filter will allow all desirable mail through and block all spam, but since the spammers are constantly fighting the filters, new rules must constantly be implemented.  Having a subscription with the spam filter vendor allows the filter to be updated with defences against new spamming methods very soon after they are discovered.  Here are some of the more common methods included in spam filters to detect and protect against spam.

Heuristics/Bayesian analysis – These intelligent filters learn and use statistics to determine if a message is spam.

Reputation – Most anti-spam vendors keep databases of known spammers and tag any mail from their domain or IP address as spam.

Phishing – The scan engine looks for links in emails to known phishing sites and tags them.

RBL – Realtime or Relay blacklist – These are third party lists of reported spammers on the Internet.  These blacklists often list IPs and domains that have temporary issues with malware, or are part of a large range of addresses.

Header checking – The scan engine compares the SMTP and MIME email addresses in the message header to make sure they match as well as other header anomalies.  Email clients set the MIME address, while the server sets the SMTP address.  Spammers use this to mask the origin of an email.

Directory harvesting – Emails sent to multiple non-existent accounts in an email domain are marked as spam.  The term directory harvesting is used because by sending large lists of names to a mail server, most will bounce back as failed, but some may not.  Eventually the sender can, by process of elimination, build a list of valid email addresses on the domain.

SPF – Sender Policy Framework – A relatively new DNS record type that defines domain names and hosts that are authorized to send email on a given domain.  Although not widely used initially this method of protecting a domain has become more common.

rDNS – Reverse Domain Name Service – Where DNS takes a domain name and translates it to an IP address, rDNS looks at the IP address and ensures that the domain name in the email header matches the domain name that points to the IP.  This identifies servers that are relaying mail for other, typically unauthorized, domains.

Blacklist – A list on the anti-spam server that is generated automatically or by user intervention that identifies domains or IP addresses that users have reported as spam.

Keyword checking – A list generated automatically or by user intervention or words that are in undesirable emails.  These words could be profanity, pornographic, pharmaceutical, or any other s.

Regardless of which type of spam filtering is used and which tests are implemented, all but the most expensive appliances require configuration and some amount of learning.  Don’t expect a product to stop all spam right out of the box.  With some adjustments and ongoing updates they are great tools to keep users productive.

I recently received some digital photos from a friend and noticed that the files were named differently from the ones from my own camera.  I was wondering why different camera manufacturers would use different file formats, so I did a little digging.

Camera manufacturers are always trying to get an edge by providing better features and image quality.  Much of the image quality is determined by the lenses, but once the picture is snapped, the image data needs to be stored.  I had a look at the specs on a high end camera, the Canon EOS 1DS Mark III digital SLR and they listed the image format as:

DCF 2.0 (Exif 2.21): JPEG, RAW and RAW+JPEG simultaneous recording possible. Multiple options for recording images on two memory cards, and onto compatible external USB hard drives (via optional Wireless File Transmitter WFT-E2A)

Since this is a high end camera, I thought that someone considering this purchase would be well up on the terminology and acronyms, so I looked at one of the consumer rated models instead.  The Canon PowerShot SX 120 IS Digital Camera had two separate specs for file format and compression:

Design rule for camera file system, DPOF (Version 1.1) compliant

Still Image: Exif 2.2 (JPEG)
Movie: AVI (Image data: Motion JPEG; Audio data: WAVE (Monaural))

Thinking that maybe it was just common for Canon to use these acronyms, I checked other manufacturers and was greeted with terms like CCD-RAW, TIFF, DPOF, and BMP.  There are some important things to look for in a digital camera, but choosing the right specs to compare may depend on your personal needs. First some basics about image files.  Image data can be stored in two very basic formats; raster and vector.  Vector graphics are more suited to line and shape drawings, including 3D images.  The basic premise of vector graphics is to define a starting point for a line, then a direction and distance.  The direction can be on two planes (2D) or three (3D).  Other parameters such as colour, fill, and weight are also defined.  Vector graphics are commonly used in computer aided design (CAD) drawings.  Raster graphics use a method similar to that of a television screen where the image starts forming by defining a single dot or pixel, then moving to the adjacent pixel and defining it.  This would continue across one dimension of the image, then move to the next line or row until the entire image is formed.  The definition would include colour and intensity for each dot.  As you can guess, vector graphics take relatively less space to store because they can define large areas with just a few parameter definitions.  Raster graphics by contrast, must define each pixel in the image.  To combat this space requirement, a number of different image formats and compression schemes have been developed.  The best quality images have no compression, but take the most room.  Because of the detail that can be stored is raster files they are used as the storage format for cameras.  Here are some descriptions that may help:

CCD - Charge Coupled Device.  This is the special device in digital cameras that convert light to electronic data.  They are an array of microscopic devices on one silicon chip and usually number in the millions.  The term megapixel refers to the number of pixels in a CCD.  More pixels allow the CCD to detect more detail in an image.

RAW - This is not an acronym.  Image data is taken directly from the CCD and stored on the storage media.  RAW images are created very quickly, but take a lot of space.  If space is not an issue, then this is the best format for a camera.

DCF - Design rule for Camera File system.  This is a standard adopted by a number of camera manufacturers to define a standard format for camera images file systems.  The standard includes folder structure and EXIF formatted files and has been adopted by most camera manufacturers.

EXIF - Exchangeable Image file Format.  EXIF adds structure and photographic information to existing image formats to standardize the file structure and help compatibility.  Information such as shutter speed, camera manufacturer and orientation are among the parameters defined.  Various versions of EXIF are in use by most camera manufacturers today.

JPEG - Joint Photographic Experts Group defined this standard for image compression.  There are a number of levels of compression available which decrease the storage footprint at the expense of image quality.  If you're looking for a camera that can store many pictures, then look for JPEG formats.  A low level of compression will retain most of the image quality with a good saving of storage space.

TIFF - Tagged Image Format was developed by Adobe Systems in an attempt to create a common image standard for scanners.  TIFF acts as a container which defined the geometry and other parameters of an image and may contain images in other formats such as JPEG.  TIFF formats with embedded JPEG give the same storage saving as JPEG, but don't add much additional value.

DPOF - Digital Print Order Format adds definitions for output to images.  These definitions may include number of copies, output image size, orientation, and title text.  This standard was developed by a consortium of camera manufacturers to simplify printing of digital images.  DPOF was not widely adopted and has become unneccessary for digital printing.

BMP - Bitmap images were developed by Microsoft as a high quality uncompressed image format.  The large size of BMP images makes them unsuitable for cameras or as images on web pages.

PNG - Portable Network Graphics files are based on the BMP with the addition of compression for use on the Internet.  The compression of PNG results in too much image quality loss for most camera use.

While there are many other formats defined and in use today, I hope this sheds some light on what to look for when choosing an image format or a camera that uses them.

Printed documents are created by attaching ink to paper.  The process involved and the quality of the output varies widely depending on the type of printer used.  Historically there have been many methods of printing and duplicating, but I will only talk about computer printers in this blog.

Printers are classified in one of three categories based on how they produce output.  These categories are character printers, line printers and page printers.  As you might guess the output is transferred to paper one character at a time, one line at a time or one page at a time respectively.  The most popular early printers were dot matrix printers.  These could be either character or line printers.  Dot matrix line printers typically had a row of small wires lined up across the width of the paper with an ink saturated ribbon between.  The ribbon is fed horizontally across the row of wires while the paper is moved vertically.  For each vertical step of the paper a combination of wires would be pushed at the paper causing the ribbon to touch and leave an entire line of dots across the page.  As the paper continues to move up the characters of the output are formed.  These printers are very fast, but are usually limited to text or basic graphics.

Another type of dot matrix printer uses a moving print head with a small number of wires (typically 9 or 24) placed vertically.  The head is moved horizontally across the paper while an ink ribbon is fed horizontally across the head.  As the head moves across the wires are fired in a pattern to create each character of text one at a time.  The head may fire as it travels in one direction or both depending on the printer and the quality of output.  After a line of text has been completed the paper is fed vertically to position for the next line.  Higher quality text could be created using a printwheel or ball printer.  These printers used the same method as many typewriters spinning either a spoked wheel with embossed characters or a gimbled ball with embossed characters.  In the case of a printwheel, a single hammer would fire to push the character on to the paper with the ribbon between leaving the character impression.  Ball printers would lift the entire ball to the paper with the correct character positioned to push the ribbon against the paper.  Neither of these types of printer could usually print graphics, but different fonts were available by changing the wheel or ball.  These are also considered character printers and all of the technologies I’ve mentioned are able to print multi part forms because they use impact to transfer the image.

Newer technologies have replaced most dot matrix printers.  Ink jet printers are character printers that are very similar to dot matrix character printers except that they force ink directly onto the paper through tiny nozzles on a moving head.   The head is moved across the paper while small dots of ink are ejected using either bubbles created by heat or electrically induced constrictions of the nozzles.  As with the dot matrix printers the dots form the characters as the head moves.  Because there is no impact, multi-part forms are not able to be printed.  The small size of the dots used to form the images allow for very high quality text or graphics to be printed.  Most inkjet printers also use multiple print heads moving together to produce full colour output.  Although there are a number of printers that use six or more heads, it is more common to have only four; black, cyan, magenta and yellow.  The cyan, magenta and yellow can be placed on the paper in close groups of dots causing your eye to see the combined colour.  This allows for a full spectrum of colour reproduction.  Although these printers produce very high quality output, they are usually quite slow and expensive to operate due to the high cost of ink.

For better speed and lower cost of operation without too much sacrifice in image quality there are laser printers.  Laser printers are considered page printers because the entire page or text or graphics are produced within the machine, then transferred to the paper.  There are a number of laser technologies currently in use, but the most common uses a statically charged rotating drum.  As the drum rotates a laser beam is fired at the surface.  A spinning mirror directs the laser from side to side on the rotating drum as it is pulsed on and off.  Where the laser hits the surface of the drum the static charge is removed.  Farther along the rotation the surface of the drum is passed by a brush of charged ink particles called toner.  The toner is attracted to the drum surface where there is no charge laying an image.  Paper is then fed between the drum and another charge which attracts to toner to the paper.  The paper is then fed between a hot roller and a pressure roller to fuse the image to the page.  Colour images are made by rotating the drum four times past four different toner brushes, cyan, magenta, yellow and black, before feeding the paper past to receive the image.  Laser printers are usually more expensive to purchase than inkjet printers, but cost less to operate over time.   

Have you ever gone to a store to buy blank recording media and wondered what all the letters and symbols mean?   I decided that it might be interesting to look at all the different types and discuss their uses.

In 1985 Sony and Philips developed a standard for writing data to compact optical disks previously developed for the music industry.  The media became widely known as CD-ROM or compact disk read only media as it could be read by an optical drive, but no data could be added.  Since the capacity of these disks were many times the capacity of floppy disks, manufacturers quickly began distributing their software on CD-ROM.  These disks were created by stamping a reflective sheet with a glass master similar to the process used to make vinyl records in the days before audio CDs.

As technology continued to develop, new CD-R or CD Recordable media became available where data could be written to a disk with a CD “burner”.  The new CD-R media had a dye film instead of a reflective film and the CD burner used a laser to alter the reflective properties of the media.  The changes, once made, were permanent making it a good format for permanent backups and small run software distribution.

Later CD-RW or CD ReWritable media and burners were developed allowing usage similar to a floppy disk with much greater capacity.  The rewritable media would allow data to be written, erased, and modified.

In a similar evolution to CDs, DVD media also made the transition to the computing world.  Originally developed as a medium to store and distribute video, the potential for data storage was quickly recognized and a consortium of computer manufacturers agreed on a standard format.  The purpose of this agreement was to help avoid a format war similar to the VHS/Betamax battle of the early 80’s.  Unfortunately after the initial DVD-R standard was created the war was started anyway with the creation of a DVD+R format which is similar to, but incompatible with, the DVD-R format.  This means that you need to make sure your burner is compatible with both formats or choose the correct media.

Like CD’s, DVD’s have a RW version and media is available for both formats; DVD-RW and DVD+RW.  These disks have a storage capacity that is 6.4 times that of a CD.

Another innovation in this technology was the creation of dual layer media.  Introducing a second dye layer to the disk allows almost twice the amount of data to be stored on a single disk.  Dual layer media is available in both write once and rewritable format.  This media is identified by adding DL to the end of the name such as DVD-RW DL or DVD+R DL.

The latest addition to this media family is Blu-Ray.  As the next generation of storage technology was being developed a “media format war” of types began again.  This time the battle was short lived and the proponents of HD-DVD conceded in February 2008 that Blu-Ray would be the new standard.  With single layer disks storing up to 25 Gb of data and dual layer storing 50 Gb, this format is suited for storing high definition video or large quantities of data.  This media is labelled BD-R for recordable media, BD-R DL for dual layer recordable media and BD-RE for erasable media that can be written to over and over.

Here’s a summary:

Media                                                   Data capacity                                     Music/Video capacity (Minutes)*

CD-R, CD-RW                                      700 Mb                                                 80

DVD-R, DVD+R,                                   4.7 Gb                                                   535/60

DVD-RW, DVD+RW

DVD-R DL, DVD+R DL,                         8.5 Gb                                                   970/120

DVD-RW DL, DVD+RW DL

BD-R, BD-RE                                        25 Gb                                                    2850/350

BD-R DL                                               50 Gb                                                    5700/700

*Note that video capacity is approximate and assumes no compression.  Compression increases the video capacity with some sacrifice in picture quality.  HD video uses considerably more space.

One last technology on note is LightScribe.  Using LightScribe media in a compatible burner, a label or design can be etched into the top of the media.  LightScribe enabled burners will be labelled on the front and include software to design the label.

On January 1st the British Columbia government imposed a new law restricting drivers from using handheld devices such as cell phones and GPS units.  The need to create this law and the associated fines might seem a bit surprising considering that 15 years ago hardly anyone had a cell phone and only a very small percentage of those who did had any kind of electronic messaging on them.  It seems that there has been a change in people's expectations when they communicate because of the increased availability that cell phones create and tolerance for someone being "unavailable" is low.  So how does someone stay connected while still being a safe and legal driver?  In short we can't yet, but there are technologies to fill some of the gaps.

Bluetooth is probably the most well known and used hands free solution for cell phones.  Originally designed as a wireless replacement protocol for RS232 serial connections, it uses multiple frequencies to transmit data making it more resistant to interference.  Since voice data is time sensitive in that the listener can hear delays and retransmissions, Bluetooth is an excellent technology to carry voice to cell phones. Many other devices such as GPS units and computers are equipped to use Bluetooth as a unified standard for communication.  Unification is the main intention of the designers as noted on Wikipedia:

"The word Bluetooth is an anglicised version of Danish Blåtand, the epithet of the tenth-century king Harald I of Denmark and parts of Norway who united dissonant Danish tribes into a single kingdom. The implication is that Bluetooth does the same with communications protocols, uniting them into one universal standard.^[1][2][3] Although blå in modern Scandinavic languages means blue, during the Viking age it also could mean black. So a historically correct translation of Old Norse Harald Blátönn could rather be Harald Blacktooth than Harald Bluetooth.

The Bluetooth logo is a bind rune merging the Germanic runes  (Hagall) and   (Berkanan)."

Many cell phones and GPS devices use speech recognition to dial numbers and enter destination addresses.  Combined with Bluetooth earpieces or microphones built into cars, they can be used with very little manual interaction.  In fact, some devices such as the Apple iPhone and RIM BlackBerry also have available applications to read emails aloud.  Speech recognition for dictating emails and text messages is in limited use on some devices as well.

I personally find it difficult not to pull out my BlackBerry when a message comes in, but the new law has not only made me think about the possible fines associated with succumbing to this distraction, but also the danger to myself and others.  If society doesn't allow us to revert to being unavailable when we're driving, then hopefully technology will quickly come to our aid and keep us safe on the roads.

For more information on Bluetooth visit Wikipedia or the official Bluetooth site below.

http://en.wikipedia.org/wiki/Bluetooth

http://www.bluetooth.com/bluetooth/