STR #3126

FLTK matrix user chat room
(using Element browser app)   FLTK gitter user chat room   GitHub FLTK Project   FLTK News RSS Feed  
  FLTK Apps      FLTK Library      Forums      Links     Login 
 Home  |  Articles & FAQs  |  Bugs & Features  |  Documentation  |  Download  |  Screenshots  ]
 

Return to Bugs & Features | Roadmap 1.3 | Post Text | Post File | Prev | Next ]

STR #3126

Application:FLTK Library
Status:5 - New
Priority:1 - Request for Enhancement, e.g. asking for a feature
Scope:3 - Applies to all machines and operating systems
Subsystem:Image Support
Summary:Image resizing algorithm
Version:1.3-feature
Created By:rokan2
Assigned To:Unassigned
Fix Version:Unassigned
Update Notification:

Receive EMails Don't Receive EMails

Trouble Report Files:

Post File ]
Name/Time/Date Filename/Size top right image
 
#1 rokan2
10:30 Sep 03, 2014
test.png
60k
 
 
#2 rokan2
10:31 Sep 03, 2014
register_images.cxx
11k
 
 
#3 rokan2
07:48 Sep 05, 2014
test2.png
227k
 
 
#4 rokan2
07:49 Sep 05, 2014
resize_image.cxx
18k
 
 
#5 chris
10:14 Sep 12, 2014
copy_bilinear_diff.txt
25k
 
 
#6 chris
10:15 Sep 12, 2014
copy_bilinear.tar
70k
 
 
#7 chris
08:40 Sep 14, 2014
copy_bilinear_diff_new.txt
26k
 
 
#8 chris
08:40 Sep 14, 2014
copy_bilinear_new.tar
70k
 
bottom left image   bottom right image

Trouble Report Comments:

Post Text ]
Name/Time/Date Text top right image
 
#1 rokan2
10:30 Sep 03, 2014
I have seen that you have applied a patch for bilinear rescalling using bilinear algorithm. I hope this wont sound as a selfish plug but please consider my attached method instead:

- It is better that the "float" implementation from 2869. Although I have called my "int" method "bilinear", it is better than that: when resampling down - all pixels contribute to the result with the same total weight-sum: each new pixel is "weighted combination" of surrounding pixels and sum of all contributions for each old pixel is equal to "1" (for instance if pixel lies on the boundary of two "new pixels" it partially contrubute to both with the ratio corresponding to crossection areas) No pixel information is lost as in "Ordinary Bilinear" method where many pixel might get (enven completely) lost if there are other pixels lying closer to the centre of new pixels. This is what Bill also mentioned that ordinary bilinear interpolation is not good enought.

As an example see attached screenshot: ordinary float bilinear image has still aliasing problems, the text resampled with my method is more readable - see especially button labels Fl_Button, Fl_Repeat_Button in the bottom right corners.

- My method should be faster as it uses "int" values than float (although I did not test it, probably slower than the nearest-neighbour though)

- I think that in the current patch with further multiplication of RGB channels by alpha is a BUG: the channels should be resampled independently so if it was premultiplied before it still stays after resample but you CAN NOT artificially make from non-premultiplied channels a premultiplied one just during resampling (imagine that you resample 1:1 - atrer resizing you get completely diffetent image because you modify RGB channels). I believe that Fl_RGB_Image specification is that RGB channels are not premultiplied but even if they were resizeng chennels independently would work just fine. Lauri mentioned that he sees problem when he loads a PNG from some memory data: I presume it is an image load bug when it converrts the PNG data to RGB image (either by assigning this data or there is a bug in Fl_PNG_Image constrictor) but either way this SHOULD NOT be corercted during resizing, resizing has nothing to do with it.


Attached you will find my little-bit cleaned-up resizing functions and a png image with comparison of all three methods.

Roman
 
 
#2 rokan2
08:20 Sep 05, 2014
Ok I stand correcred, I have modified my file to do alpha-premultiplication, see latest version of the file ad also see test2.png

I have question about Lauri's implementation : Do you "unmultiply" the values after mixing - because I cant see it in your's code. Also look at the "test2.png" - your resized image looks "bigger" than the original and my images. Maybe this is the alpha artifact of the semi-rtansparent edges and missing "unmiltiplification": because RGB values are not divided by resulting alpha (valuesl smaller than 1), they have lower RGB values than they should (darker) and cause this artifact?

I have also measured speed and these are my data (10 times, values in seconds - Intel Core5 3.3GHz)

1) UP resizing from 497x497 -> 2301x2301 (ten times)

Nearest  0.198
Lauri's  2.529
Mine     2.130


2) DOWN 2301x2301 -> 497x497

Nearest  0.016
Lauri's  0.225
Mine     0.647


- When resizing up, mine is only slightly faster than Lauri's. This is probably because float operation is pretty fast these days (especially on Intel), I have to use also some 64bit integer operations to avoid overflow for big images.

- When resizing down Lauri's is faster. This is because its implementation performs number of operations proportional to number of TARGET pixels where as for mine it is proportional to number of SOURCE pixels (but the speed comes at the expense of quality because info from (most) of the pixels gets lost in the process). The numbers might also change if Lauri adds this "unmultiplication" which would involve relatively expensive division operation.

R.
 
 
#3 cand
08:40 Sep 05, 2014
Yes, I added the missing un-multiply in a recent commit, please check the latest code. With it, the 1:1 scale scenario also produces correct values.  
 
#4 rokan2
11:33 Sep 05, 2014
OK, just consider that you might be dividing by 0  - alpha is very often 0, although in the case when alpha is 0 RGB channels dont matter.

Now new testing gives me times for the same resizing as above (gcc, -o3):

UP:

Nearest 0.194
Lauri's 2.512
Mine    2.134

Down:
Nearest 0.015
Lauri's 1.903
Mine    0.647

So this division is quite expensive and makes it slower (I presume that the second division by constant 255.0 is optimized by the compiler to multiplication by the inverse so this won't help).
 
 
#5 cand
02:28 Sep 06, 2014
As the quality is now up to par, and the speed is nice, I'm open to replacing the float one with this, under the same FL_SCALING_BILINEAR option.  
 
#6 AlbrechtS
04:51 Sep 06, 2014
Changed title ("summary"), fixed typo. This is to make sure that searching for "image" will find this STR.

+1 for replacing the existing algorithm with this one, if there are no obvious reasons to have both.

Performance seems to indicate this one is faster, and if the quality is also better (or equal), I don't think we need the float algorithm any more.
 
 
#7 rokan2
14:41 Sep 06, 2014
Just for the record performance-wise it really depends on the particular platform, image sizes and maybe also on compiler: just tested it at home on older Core2 and resizing up and down up to about 0.5 is still faster but below 0.5 Lauri's start to be faster  - which is quite logical as more operations are involved: the quality is at the expense of the speed...

R.
 
 
#8 chris
10:14 Sep 12, 2014
As no one has yet come up with a patch to integrate this into FLTK, I'll go ahead and share my take on it...

My goals have beeen:

a) Keep ABI/API compatibility
b) Keep the bilinear algorithm in a separate source file
c) Change as little as possible

ad a) I introduced a protected static method Fl_RGB_Image::copy_bilinear().
      (I didn't want to use a globally visible static method).
ad b) the source has been called 'rgb_image_copy_bilinear.cxx' and
      contains Roman's implementation plus the Fl_RGB_Image::copy_bilinear()
      implementation as wrapper to the resampling methods(*).
ad c) Fl_Image.cxx and Fl_Pixmap.cxx(**) use the this method in their
      bilinear case.


(*)  The source is unchanged with the exception that the U32 typedef
     collided with a define in config.h, so I had to use another typedef
     - and of course the wrapper method.
(**) This was easy to achieve with the recently added pixmap to rgb
     conversion through Fl_RGB_Image(Fl_Pixmap *).

I also tweaked the pixmap_browser test program to change the scaling method and resize mode so differences can be studied better.

The patch is available either as a diff ('bilinear_copy_diff.txt' or a tar with all changed files ('bilinear_copy.tar').

Cheers,
Chris
 
 
#9 cand
04:44 Sep 14, 2014
Mainly style comments about the patch:
- you added/moved ifs without changing the indentation accordingly
- the else in Fl_Pixmap is wrong style, should be "} else {"
 
 
#10 chris
08:40 Sep 14, 2014
- Corrected indentation in Fl_Image.cxx (including leading tabs, that have been there before my patch)
- Corrected if .. else indentation in Fl_Pixmap.cxx
- Simplified the change in Fl_Pixmap.cxx

==> copy_linear_diff_new.txt, copy_linear_new.tar
 
 
#11 cand
23:30 Sep 14, 2014
The same indent issue is still in Fl_Pixmap.cxx?

Also, the anonymous namespace is an open question whether that's allowed, I have another patch waiting on that one. However here it's not needed and the normal "static" qualifier can be used, as these are C functions, not classes.
 
 
#12 chris
00:02 Sep 15, 2014
Concerning Fl_Pixmap: I started to to correct Fl_Pixmap indentation, but it had no end, because I tried to avoid it by moving the bilinear case to the top, then got stuck with the comments below not fitting well any more and anyway there are a lot of tab characters in Fl_Pixmap.cxx and the patch got more and more complex to overview. So I think a "refactoring STR" for Fl_Pixmap would be more adequate, once (and if at all) this patch is in place.

Regards,
Chris
 
 
#13 AlbrechtS
05:11 Nov 21, 2014
Can we close this STR?

A new bilinear image resizing algorithm has been implemented in FLTK 1.3.3.

Is there still need for any of the patches included here?

Sorry, I'm too busy to read all these patches and decide whether they are still useful. So anyone who thinks they are still applicable should add a comment.

If there are issues other than the bilinear image resizing please file another STR and append new patches.

If there is no reaction this STR will be closed within the next 7~10 days.
 
bottom left image   bottom right image

Return to Bugs & Features | Post Text | Post File ]

 
 

Comments are owned by the poster. All other content is copyright 1998-2022 by Bill Spitzak and others. This project is hosted by The FLTK Team. Please report site problems to 'erco@seriss.com'.