[PNG icon] PNG Support in Mozilla [OLD!]

(Last update: 2000-10-21 )

Intro

This page currently contains Greg's (incomplete, out-of-date) notes on the basic flow of control through Mozilla's ImageLib and various front ends when decoding and displaying a PNG image, with special attention to alpha-channel handling. Some of it still applies to Classic Mozilla, but Greg updated most of it for SeaMonkey/Gecko/Raptor/NGLayout (a.k.a. proto-Navigator 6.0, basically rewritten from the ground up).

Classic Mozilla didn't do any compositing on its own; it left that to various platform-dependent API calls in the front ends, which is why it was limited to binary transparency. (For example, X does not provide any APIs for dealing with an 8-bit alpha channel, only 1-bit transparency masks.) SeaMonkey includes the new layout engine (at one time called Raptor) that supposedly allows an image-rendering component to figure out what's beneath the image frame and do proper alpha-blending. Certainly this appears to be the case in Demo 10 bundled with SeaMonkey, wherein a partially transparent animated GIF (that is, using both GIF binary transparency and CSS partial transparency) is displayed correctly on top of an animated background image.

Current Status

On 13 April 2000, Pam Nunn checked in some code that significantly improved the state of PNG transparency support in Mozilla. At a minimum this provided random-dithered transparency, a way of approximating a full alpha channel on any device or platform that can do bitmasks. That code was provided by Glenn Randers-Pehrson, and these screen shots demonstrate that it works pretty well--much better than simple thresholding, anyway, though not nearly as nice as true alpha-blending.

Even better than that, Tim Rowley managed to get full alpha-blending working on Unix (albeit via a hack), and it looks fantastic! (Well, at least on a 24-bit X display; 8-bit displays are likely to be another story altogether.) Nevertheless, it's a huge improvement over the previous status. Outstanding!

On top of that, Simon Fraser did the same for Macintosh, and as of 20 April 2000, 8-bit alpha blending was enabled on all three main platforms. Unfortunately the Windows version still needs work, but it's getting there.

Previous Status

[Greg speaking in the first person for a change...]   In mid-January I wrote, ``I haven't quit, but I'm not getting much done, either.'' A few days later, I did quit, however. I felt this was only fair to the Mozilla folks, given that I had not made any progress in four months and hadn't even had a working build in two months.

Why the lack of progress? I had devoted huge amounts of time to Mozilla all through the spring and summer of 1999, 95% of which was spent simply trying to stay in place. Unlike most Open Source projects--which tend to become more portable and inclusive as time goes on--Mozilla tended in the opposite direction, at least through 1999. It was never particularly friendly to Linux/libc5 systems, and as of the Necko landing (new networking code) at the end of July 1999, the decision was made to shut out Red Hat 5.2 systems, too (that is, glibc 2.0), and support only glibc 2.1 on Linux (e.g., Red Hat 6.x, etc.). That's perfectly understandable for a project that is 99% underwritten by a large corporation with shareholder interests to worry about, but it's rather hard on part-time developers who can't afford the time required to switch to another distribution or to set up a second development machine. (The ironic part is that I switched to a glibc 2.1 Slackware setup in mid-January 2000, so most of the build problems I wrestled with in the previous nine months should have evaporated. But by then I had used up all of the time I could afford to spend on Mozilla. Sigh.)

The other half of the story is that the code base is extremely large, and trying to figure out how to get from here to there is quite a task all by itself. With regard to alpha-transparency support in PNG images, the kicker is that I was never able to find my way from the high-level view of web-page elements (starting with, say, the ``kids'' in view/src/nsView.cpp) to the low-level code in gfx/src and ImageLib. The high-level code is the only place alpha-compositing can be done in a platform-independent manner--i.e., not just on Macs and Win32/DirectDraw systems but also on Unix/X and others--but I never got very far by working backward from the low-level code. (Is an image container a ``kid''? How can one tell? Beats me...)

Add to that the fact that the persistent PNG interlacing bug cannot be squished via a single bitmask (which is all Mozilla currently supports), and we again run up against the same root problem. That is, even though I understand the PNG code pretty well, the limitations that stand in the way of implementing full alpha transparency and of fixing the interlace bug all live elsewhere, and I simply don't have the time to try to understand those parts of the code.

Finally, I never intended this to be a one-man project, but that's pretty much the way it turned out. I absolutely don't blame anyone for shying away from the Beast, but it would have been nice to have had a little help once in a while. Tracing through the code is something even a user of a non-Win32/Mac/RH6 system could have managed (thanks to LXR). Oh well.

So I've been devoting my energies to PNG and other Open Source projects were I can actually be productive, like gif2png, tiff2png, pnmtopng, rpng/rpng2/wpng, libpng MMX optimizations, intel2gas, pngcheck, gd, the PNG and MNG web sites, and (gosh!) even family and work. I briefly toyed with the idea of implementing at least dithered alpha (see immediately below) in Mozilla, and as of April 2000 there's some talk about supporting that in libpng itself (png_set_dithered_alpha() or similar), but I've made a conscious decision not to kill myself for the sake of Netscape's timetable--or even my own past promises (sorry). While I don't agree with everything Jamie said, he did have a couple of valid points. Nevertheless, I'm trying hard not to become apathetic (or, even worse, bitter) about the whole thing.

Comments welcome, of course.

[the following section is almost two years old...pardon the mess]

Also, Greg had a minor revelation while standing in line for the Future of Linux panel discussion on 14 July 1998: as a stopgap measure, why not simply replace the current alpha-to-bitmask conversion code with a ``smart'' version that uses Floyd-Steinburg dithering (or some other error-diffusion method)? The result is still a bitmask, but it's a bitmask that simulates an alpha channel halfway decently, at least allowing the general flavor of RGBA images to shine through. And since it involves absolutely no changes to the front ends, it's something that can possibly be integrated into the Communicator tree prior to 5.0. Greg will be looking into this as soon as the book is done.

See http://www.mozilla.org/ for pointers to the full Mozilla source code, documentation, schedules, projects, etc. All of the code is web-accessible from http://lxr.mozilla.org/; the core browser itself (SeaMonkey) is at http://lxr.mozilla.org/seamonkey/ and includes search facilities. Also check the Mozilla owners list for contact information on PNG and related functions in Mozilla. Most PNG-related discussion (not much) occurs on the png-implement mailing list; see the Miscellaneous Links page for subscription info. Note that the old LXR site (http://cvs-mirror.mozilla.org/webtools/lxr/) is no longer kept up to date, at least not at its top-level page.

Finally, here are some convenient links relevant to the code, courtesy of Pamela Nunn and others:

Here are some of the reported bugs that are related to PNG support:

These are the MNG-related bugs:

Build Notes

Greg's Mozilla build notes have moved to a separate page.

Relevant Source Files

Note: Some of this is out of date at the moment; it applied to MozillaClassic, not SeaMonkey. Front-end stuff that used to live under cmd now lives under gfx/src. 

modules/libimg/png/				libpng 1.0.2 (originally 0.95b)
modules/libimg/pngcom/				libnspng (XPCOM interface to libpng)
modules/libimg/mng/				libmng 0.9.0
modules/libimg/mngcom/				libnsmng (XPCOM interface to libmng)
modules/libimg/src/				main ImageLib routines (generic)

modules/libimg/public/ni_pixmp.h		NI_TrueColor type, etc.





modules/libimg/pngcom/ipng.cpp			main ImageLib interface to libpng

    il_png_init (ic)
      [ ic->src_header->color_space->type = NI_TrueColor ]
      [ ic->src_header->color_space->pixmap_depth = 24 ]

    il_png_write (ic, buf, len)
      [ call chain:
	ImageNetContextSyncImpl::GetURL() in gfx/src/nsImageNetContextSync.cpp or
          [ this is where the error-return code ultimately gets ignored ]
	ImageConsumer::OnDataAvailable() in gfx/src/nsImageNetContextAsync.cpp
          [ this routine also ignores error-return codes ]
	  NetReaderImpl::Write() in ilNetReader.cpp
	    IL_StreamWrite() in if.cpp
	      PNGDecoder::ImgDWrite() in nsPNGDecoder.cpp
		il_png_write() in ipng.cpp
      ]
      png_set_progressive_read_fn (png_ptr, buf, info_callback, row_callback,
        end_callback)		[ first time only ]
      [ png_ptr->io_ptr = ic ]
      png_process_data (png_ptr, info_ptr, buf, len)

    il_png_delay_time_callback (closure)	[ commented/ifdef'd out ]

    il_png_complete (ic)	[ called by PNGDecoder::ImgDComplete() ]
      ic->imgdcb->ImgDCBHaveImageFrame()
      [ also other stuff, but ifdef'd out (WE_DONT_HAVE_SUBSEQUENT_IMAGES) ]

    il_png_abort (ic)	[ called by PNGDecoder::ImgDAbort(); frees memory ]

    info_callback ()
      [ various libpng set-transformation calls ]
      png_read_update_info ()
      [ former contents of png_set_dims ]
      [ possible bug:  sets width, height based on png_ptr values ]
      [ possible bug with gray+alpha images (channels == 2) ]
      [ uses both png_ptr->io_ptr and ic:  identical ]
      il_create_alpha_mask (io_ptr, 0, width, height)	[ png_png.cpp ]
      il_init_image_transparent_pixel (ic)		[ if.cpp ]
      il_size (ic)					[ if.cpp ]
        [ this is where the max-image-size limitation lives (8000 pixels) ]
      il_setup_color_space_converter (ic)		[ color.cpp ]

    row_callback ()
      [ sets ic but doesn't use it ]
      il_emit_row (png_ptr->io_ptr, 0, new_row, 0, png_ptr->width, row_num,
                   1, ilErase, png_ptr->pass)		[ scale.cpp ]

    end_callback ()
      [ sets ic but doesn't use it ]
      il_emit_row (png_ptr->io_ptr, 0, new_row, 0, png_ptr->width, row_num,
                   1, ilErase, png_ptr->pass)		[ scale.cpp ]

    il_png_error_handler (png_ptr, msg)		[ callback for libpng longjmp() ]




modules/libimg/src/scale.cpp			most of the good stuff!

    il_timeout_callback (closure)	[ closure == ic, apparently ]
      il_flush_image_data (ic)
      FE_SetTimeout(il_timeout_callback, ic, delay)

    il_partial (ic, row, row_count, pass)
      [ ic->multi:  multi-image GIF, for example? ]
      [ if no delay ] il_flush_image_data (ic)
      FE_SetTimeout (il_timeout_callback, ic, ROW_OUTPUT_INITIAL_DELAY)

    il_flush_image_data (ic)
      IMGCBIF_UpdatePixmap(ic->img_cx->img_cb, ic->img_cx->dpy_cx, image, 0,
                           row, img_header->width, end_row - row + 1);
      IMGCBIF_UpdatePixmap(ic->img_cx->img_cb, ic->img_cx->dpy_cx, mask, 0,
                           row, mask_header->width, end_row - row + 1)
      [ IMGCBIF_UpdatePixmap() is defined in modules/libimg/public/MIMGCBIF.h as
        (effectively) ic->img_cx->img_cb->vtable->UpdatePixmap, which points at
        _IMGCB_UpdatePixmap() in:
		cmd/xfe/images.c		(Unix/X)
		cmd/macfe/central/mimages.cp	(Macintosh)
		cmd/winfe/feimage.cpp		(Windows)
      ]
      il_pixmap_update_notify (ic)			[ if.cpp ]
      il_progress_notify (ic)				[ if.cpp ]

    il_scale_RGB_row (src, src_len, dest, dest_len)
      [ scales row of RGB pixels ]

    il_scale_CI_row (5 args..., transparent_pixel_color)
      [ scales row of indexed-color pixels ]

    il_alpha_mask (many args)
      [ this is where the alpha-dither-to-bitmask trick goes ]

    il_generate_scaled_transparency_mask (many args)
      [ used for GIF (binary) transparency; also PNG binary transparency? ]

    il_reset_background_pixels (5 args)
      [ apparently for dithered pseudocolor displays only ]

    il_generate_byte_mask (5 args)
      [ for pseudocolor displays only? ]

    il_overlay (5 args)

    il_emit_row (many args)	[ called by row_callback(); 400 lines ]
      il_generate_scaled_transparency_mask (many args)	[for GIFs]
      il_alpha_mask (many args)				[for alpha PNGs]
      [ this is where the interlace bug lives ]
      [ tons of other stuff ]
      il_partial (ic, drow_start, row_count, pass)




modules/libimg/src/if.cpp

    IL_StreamWrite (ic, str, len)	[ called by something in netlib? ]
      il_image_stopped (ic)
      (*ic->write) (ic, str, len)	[ il_png_write(), etc. ]
      il_progress_notify (ic)

    IL_StreamFirstWrite (ic, str, len)
      il_type (ic->type, str, len)	[possibly override MIME type ]
      [ ic->write = il_png_write() or il_jpeg_write() or ... ]



gfx/src/nsImageRenderer.cpp
gfx/src/nsImageRequest.cpp
gfx/src/nsBlender.cpp



gfx/src/gtk/nsImageGTK.cpp



gfx/src/windows/nsImageWin.cpp



gfx/src/mac/nsImageMac.cpp

Obsolete Source Files 

modules/libimg/pngcom/png_png.cpp		(originally based on libpng's example.c)

    [ now part of ipng.cpp ]



cmd/xfe/images.c

    _IMGCBIF_NewPixmap ()
      [ also hardwired to assume mask_depth == 1 ]

    _IMGCBIF_UpdatePixmap ()
      XPutImage ()

    _IMGCBIF_DestroyPixmap ()

    _IMGCBIF_DisplayPixmap ()	[ 150 lines ]
      fe_DrawMaskedImageWithClipRegion (dpy, drawable, img_x_pixmap,
        mask_x_pixmap, width, height, img_x_offset, img_y_offset,
        x_offset, y_offset, fe_drawable->clip_region);

    fe_DrawMaskedImageWithClipRegion ()
      FE_ForEachRectInRegion (clip_region, fe_FillPixmapRectFunc, &fill_closure)

    fe_FillTiledPixmapRectFunc ()
      XP_OffsetRect ()
      XP_IntersectRect ()
      XFillRectangle ()



cmd/xfe/region.c

    FE_ForEachRectInRegion (region, func, closure)
      func (closure, &rect)	[ fe_FillPixmapRectFunc() in images.c ]




cmd/winfe/feimage.cpp

    _IMGCBIF_NewPixmap ()

    _IMGCBIF_UpdatePixmap ()

    _IMGCBIF_DestroyPixmap ()

    _IMGCBIF_DisplayPixmap ()
      displayContext->fe.cx->DisplayPixmap ()		[ cxdc1.cpp ? ]




cmd/winfe/cxdc1.cpp

    CDCCX::DisplayPixmap()
      [ appears to be actual compositing code ]
      MaskBlt ()					[ Windows call? ]




cmd/macfe/central/mimages.cp

    _IMGCBIF_NewPixmap ()

    _IMGCBIF_UpdatePixmap ()

    _IMGCBIF_DestroyPixmap ()

    _IMGCBIF_DisplayPixmap ()
      [ no transparency support at all?? ]

Here are some related PNG pages at this site:

[primary site hosted by SourceForge] Last modified 14 March 2009.

Copyright © 1998-2002 Greg Roelofs.