• Media Player 01.11.2008

    As you following my blog knows, I have dug down deep into the coding of the GUI. I started there since I deemed that to be the most risky part given Java’s tendency for being slow, unless you know exactly what you do and on which platform. I have coded Swing for a long time so I thought that the problems would be quite a few but I was was confident I could code around them.

    Seems I was wrong, or at least it looks like that right now. Congratulations to those who left comments in the line of “Why Java since it’s so slow and unreliable on the desktop”. The reasons are complicated but I will try to explain them below. Why? Because maybe if I write them down I can solve them? ;)

    First I made a GUI with a few cool, but subtle, effects. I have created the carousel component, the totally customized JScrollBar and both optimized glow and shadow effects. Everything was really fast and smooth, much because of the hardware accelerated image blits and scales. I am very happy with the carousel, it is every bit as fast as iTunes’ Cover Flow.

    The second phase was about getting the side docked pullout sidebars to work. I had several fallback options for this that all should look the same as in the screen shots. And since I develop on the Mac using Java 1.6.0_07 everything went really smooth here as well.

    To enable translucency you just set the background color to a color that have transparency (e.g. new Color(0, 0, 0, 0) and the rest is handled by the Mac Java implementation. I draw the shadows myself since I needed them to look the same for all platforms. It is easier if I make everything as one big window with the pullouts as normal components (and not heavyweight windows). On the Mac this work beautifully, fast and totally seamless.

    Then I switched to test on Windows Vista and XP. There were some initial problems but that was mostly because of me and small inconsistencies that I wasn’t aware of. Like that you need to constantly poll for a new Graphics object from the component you want to animate on Windows. In OS X you could hold on to the same one. No biggie.

    The trick of just setting a background color to get transparency doesn’t work in Windows, not even u10. Not sure why since it is such an obvious way to do it. Instead you have to add two lines, one which is in the protected sun libs. This makes this version update 10 and later only, unless of course I do it using reflection. The lines are:

    window.setUndecorated(true);
    com.sun.awt.AWTUtilities.setWindowOpaque(window, false);

    With those lines pixels that aren’t painted with an opaque color will have the background show through.

    But.. The performance totally went out the window (pun intended). With that last line in the code the GUI turned to dough though. Even selecting a row in the JTable was painfully slow and with a very noticeable lag. Unusable even. After an hour of investigation I found out, not from the documentation mind you, that when translucency is enabled the whole component tree of the frame will be repainted for every little local repaint!!! For instance, when the cursor blinks in a JTextField the whole frame will be repainted. How stupid is that! And not only that, it will be repainted to a buffer and then that buffer will be software copied to the screen! No wonder it was slow..

    OK, I fired up mail to let my favorite Sun engineers know that Java 1.6.0_10 was not bug free just yet. :) I was surprised that they told me that this was the intended behavior! To make things worse they first even tried to convince me that it had to be this way because of how windows works. But after a few emails back and forth it became clear that it is Swing that has a problem with rendering to a translucent back buffer and there were no resources to address that. Bummer… It “may or may not be fixed in an update”…

    So, I though that, well, me being a self appointed Swing/Java2D expert I could make my own back buffer and render to it. If I create my own RepaintManager I can intercept the small repaints and only repaint what is need to my own buffer. Then copy that buffer to the Frame, which in turn will be copied to the screen by Swing. But first I thought that I might do some performance testing, to be sure. It turns out that even a GUI that takes 0.0 ms to render is too slow for any window larger than 800×800 on my 2.4GHz one year old computer. So, even if I made the perfect algorithm that was infinitely fast it wouldn’t be enough. Just add a single JTextField to a translucent window of some size (like 1280×1024) and try to edit it… translucent windows in Java makes GHz feel like MHz. :) Dead end.

    OK, being an expert and all I was not about to give up just yet. I praised OS X’s version of Java for not having any problems in the first place and took a warm shower, which is the place where most of my ideas come to. Ping! New idea!

    I can use the normal window decorations (will look worse, but still ok) and have tool like dialogs for the pullouts. I began testing and at first it looked like it could be done. But it couldn’t… It turns out that there is no way to control the z-ordering of windows/frames/dialogs in Swing. The only control you have is that you can set a Window to be “alwaysOnTop”. Not very usable since that would disrupt the interaction with the other windows on the user’s desktop. Then I tried all the modality modes. You can actually make a tool window float on top of the main window, but not below it! I must have it below since the OS drawn shadow around the Window must be on top of my pullout or it will look very weird. Also, as soon as I pressed the pullout window the main window’s title bar would deactivate which is not the intended behavior. So, dead end there as well and another few hours wasted. And I was out of hot water to get new ideas (OK, not really. (About the water)).

    This is where I am now. At a dead end. I could of course release only for the Mac, but no.. That would go against the whole thing of sharing music with your friends, not many of them have macs…

    Actually when I was writing this I came up with another idea. Not sure it is viable but I might try it. I can set the main window and pullouts to clip with round rectangle shape. No window decorations. That means no problem with z-order or deactivated title bars since I draw the title bar and the windows will not overlap. Of course this means that there won’t be any shadows (not an option) but that might be solved with a kind of shadow-only window that has those much needed translucent pixels. Since those windows aren’t changed much, and they don’t host that many pixels in the first place it might actually work. I’ll think about it some more…

    A lot of wining, I know, but I actually thought Sun would step up to the plate and make Java a viable choice for the desktop. And when I mean the desktop I’m not talking boring enterprise apps with crappy GUIs, I’m talking media players and widgets.

    To add insult to injury it turns out that resizing windows on at least Vista turned sluggish with the jump from 1.6.0_07 to _10. Look at this post for more info: java.net forum link

    Cheers,
    Mikael Grev

    ps. For those of you that come here for the screenshots, if you have managed to get through my ramblings here’s how the Media Player looks right now on the Mac. Everything, including the shadows, is drawn with Java2D (except the album art picture of course) and it is really really fast. Even the volume control is working… :)

    Posted by mikaelgrev @ 12:32 pm

    Tags:

  • 23 Responses

    WP_Modern_Notepad
    • fanguad Says:

      Have you tried using JDIC to handle your transparency? I don’t know what mechanism it uses under the hood exactly (I think a native library for each system), but I’ve achieved reasonable performance in a media player application of my own. My window is a lot smaller, and there are fewer effects, but you might want to give it a try.

      https://jdic.dev.java.net/

    • Mikael Grev Says:

      Hello fanguad,

      I can’t actually find any info on JDIC and translucency… https://jdic.dev.java.net/nonav/documentation/javadoc/jdic/index.html does not reveal anything regarding this afaik.

      Note that I don’t want to make the window translucent as a whole, I want the option to make any part of the window partly translucent.

      There is a “Robot” trick that one can use which basically copies the background to your app and you can use that to simulate transparency. But it looks awful when you drag the window… And you need full access to the system to use it, no sand box.

    • Steven Says:

      So, when can we download and try this out!
      I’m excited

    • Mikael Grev Says:

      Steven,

      Well, that depends on if I get the new idea to work. A.t.m. it looks promising but I don’t like the many different (unnecessary) windows…

    • fanguad Says:

      oh, my bad. not jdic, but jna instead. Take a look at the Shaped Window example.

      https://jna.dev.java.net/

    • Mikael Grev Says:

      fanguad,

      Shaped windows actually works like a charm both on the Mac and u10 on Windows. It’s just that they are not anti-aliased and they have no OS provided shadow.

      I do however think I have found a very good solution… All it took was another shower! Stay tuned.

    • Andres Almiray Says:

      Something tells me this project has kept you pretty wet and clean ;-) Thanks for pushing the limits of Java2D/Swing!

    • Christophe Says:

      yup, “per pixel” transparency is possible with jdic, see
      http://swing-fx.blogspot.com/2008/02/per-pixels-transparency-with-jframe.html

      but it as some drawback (set cursor does not work, strange bugs appears …), so I would love to hear about your solution :)

    • Christophe Says:

      oh, and nice work by the way !

    • mats Says:

      You should have a look at these two blog entries (if you haven’t already):

      Discusses the AWTUtilities methods
      http://www.pushing-pixels.org/?p=260

      Discusses soft clipping (to achieve anti-aliased edges):
      http://www.pushing-pixels.org/?p=272

      However, it seems that most of these methods will turn off hardware acceleration which makes them pretty much useless for most scenarios. One noteable exception is the AWTUtilities.setWindowShape(Window, Shape) method which does not seem to have serious performance problems. I have successfully used this method to get rounded edges for my frame. (It does, however, cause a slight performance decrease when resizing the frame)

      I am sorry to say that the effects you want to achieve are most likely not possible in Java with acceptable performance.

    • not!an!exit Says:

      Have you tried using SWT instead of Swing? It looks better and runs faster. The treade off: platform-specific libraries. Definitely woth it, if you ask me…

      Anyway, I’m very excited about this project and will absolutely check it out.

    • fanguad Says:

      You can get anti-aliasing with the shaped windows – you just have to make sure to turn on the appropriate rendering hints when you draw your shape. It does support per-pixel transparency as well. I haven’t tried my application on a Mac yet, so I don’t know about shadows – but remember that you may not want shadows on Windows anyway – even if they look cool, they’ll stick out like a sore thumb.

      I’m also eager to hear about the other solution you found.

    • Swing links of the week: November 2 : Pushing Pixels Says:

      [...] Grev has started writing the media player, and has hit a serious performance issue with translucent windows in 6u10 [...]

    • Mikael Grev Says:

      not!an!exit,

      SWT is not automatically more performant. If you use their standard widgets they do feel snappier, but then Swing’s performance for those “normal” type of applications if good enough by far. Swt does not have such a good backing 2D library. Java2D is the king to beat there.

      fanguad,

      Shaped windows sends the shape to Windows, so you don’t draw it yourself. And it is not anti-aliased. If you turn on per-pixel transparency you can get aa shapes. That is what I did and when the performance got so extremely bad…

    • not!an!exit Says:

      @Mikael Grev
      That’s a good point about Java2D, never thought about that. So if I get you right, you would get not-that-fast window rendering (as with SWT) but more powerful graphics inside the window.

    • Mikael Grev Says:

      not!an!exit,

      The window rendering, if you are talking window decorations, are always drawn by the OS, even is Swing. So there’s no difference there. I don’t use the windows decorations though, as I provide my own.

      The “only” thing SWT is faster at is rendering the widgets. That, and it has a cleaner API, mostly since it was created with a more minimalistic goal and because it is newer. SWT’s API is lean and mean, though it is missing some of the flexibility that Swing has.

      Unless you want very specific stuff from SWT, like a 100% accurate file chooser, I would not use it. I like it, but Swing, which is provided by the JDK, carries no extra weight and it is fast enough.

    • Dmitri Trembovetski Says:

      Just to clarify – the slowness _is_ because of the way per-pixel translucency API on Windows platform (aka layered windows) works.

      As you said yourself, the Java2D rendering itself takes very small amount of time so it really doesn’t matter if you repaint a single pixel or your whole frame.

      But because of the way layered windows are on Windows we must upload the whole buffer every time. This is what kills performance.

      So even if we did use the ‘normal’ Swing backbuffer instead of a fake one we use for enabling translucency support we would still have to upload it all on every repaint.

      For those suggesting JNA – save your time, they use exactly the same native API as AWT for enabling translucent windows. There’s no magic there.

      I would also suggest trying on a few more systems – at least more than the single box you apparently have.

      Dmitri
      Java2D Team

    • Mike P Says:

      Mikael

      But, would SWT provide anything that might allow you to create a GUI, perhaps not as you originally had planned, but something still providing a rich application experience? I believe there are some graphics capabilities, although not as robust as Java2D. (I’m not overly familiar with SWT, but it is an interesting idea to try to create the GUI with it to see how it compares)

      On an unrelated note, I thought I read once that you were going to port MiG Calendar to SWT (it was a while ago that I thought I read that someplace..and I might be mistaken)

    • Mikael Grev Says:

      Mike, Swt is fine, though I don’t see any advantages compared to Swing today. There has been in the past, but not now. Unless you have specific needs of course.

      There are those that want MiG Calendar for SWT. Though the effort to port it would not outweigh the costs since SWT is so much “smaller” than Swing..

    • Mike P Says:

      Mikael, I do agree that SWT is smaller of an install base. I primarily use Swing, though every once in a while I revisit SWT to see how it has progressed. I have to admit, the Browser widget is great and I wish we had something similar to it as part of Swing..

    • Achaiah Says:

      Mikael,

      Have you looked at JXLayer from swinglabs.org? I don’t know if it will do the trick you want but it’s another thing to look at. Perhaps you could “fake” the current look by making a larger window with transparent sides until someone wants to pull out a sidebar which could be a JPanel resting underneath the main JPanel? You could use some custom painting to place a shadow on the sidebar. Just thinking out loud… :)

      Also, way back when, I experimented with SWT to get some transparent windows of various shapes and it seemed to work OK. It was buggy but I imagine they’ve improved it since then.

      Btw.. I’m really impressed with the carousel look/functionality. Any way we can see the code?

    • HFernandes Says:

      Well, I always thought that com.sun.awt.AWTUtilities.setWindowOpaque(window, false) lack of performance was a bug! Thank you for showing that to Sun engineers.
      Just add 100 buttons to a JFrame and then set the frame window translucent. Then try with Nimbus LAF and compare to Substance. The results are curious.

      About your Media Player – I hope this is not your final Chapter! Good work.

    • Creative Creations » Blog Archive » Java Media Player - First GUI Demo Says:

      [...] performance problem I reported in the last blog is all but gone. It’s not like the problem has been solved from Java’s side, I have [...]