Archive for March, 2012

Finding your passion – intense suggestion found in blog post comment

March 21, 2012

from a post on about finding your  passion there is a comment about a technique for finding your passion that is super intense but fascinating sounding:

And by the way, some folks commented on the fact that they don’t really understand what “following your passion” actually means. To me, those people haven’t yet found theirs or they would know. And here’s a way to find out…this comes from relationship guru David Deida…Sit or lie in a room very quietly and think about what it is you want to do with your life. You can only get up for from where you are for 3 reasons…

1) bathroom breaks
2) food
3) You have thought of something so powerful that you CANNOT NOT do it! Many things will go through your mind during this time (which can take days by the way) but there will only be one or two that will NOT let you stay still any longer because you MUST start doing it!

Try it out!


I haven’t tried this.  It reminds me of the “go to a cabin in the woods without any technology besides a drawing notebook and some pencils and stay there until you’ve thought of something…” and ends like the suggestion above.  The suggestion above is easier for most people to actualize and so in that way it’s compelling.


Xcode 4 – some thoughts

March 14, 2012

So I’ve been as frustrated as anyone by the wide array of problems with Xcode 4, from frequent crashing to painfully long slowdowns while it does something important that it doesn’t bother to tell us about, etc etc.

I even tweeted a couple of times in frustration.  Hopefully I kept from being too harsh unlike some I’ve seen (apologies if not).  It seems easy for people (me included when in the midst of deep frustration and deadline pressure) to forget that there are good, hard-working people behind Xcode and the development tools.  The ones I know certainly care a great deal about making these tools solid and excellent.  I appreciate their efforts.

I’ve used pretty much every public developer tool from Apple for the Mac, with the exception of the Macintosh Programmer’s Workshop (is that what it was called?) that ran on the Lisa (I was a  year too late for that one).  And a ton of non-apple tools as well (Rascal, Consulair C, MacASM, MPW, Lightspeed Pascal, Lightspeed C, THINK Pascal, THINK C/C++, every version of CodeWarrior, SmalltalkAgents, Prototyper, ObjectMaster, C Master, etc etc).  Even did a bit of early Objective C development with the precursors to todays Mac OS X tools on the first NeXT cube.  In all that time I can’t quite remember a less stable set of of development tools than Xcode 4, with the possible exception of THNK C++ 1.0.  On the one hand that’s troubling, on the other, it means things have been very good for us developers for a long long time.

One tweet I made was along the lines of “Apple should spend 1% of it’s cash hoard on QA for Xcode & Developer tools.”  Other developers seemed to like that sentiment given the retweets and “+1” activity it got.   Wasn’t the nicest comment I’ve made and this is the one that afterwards made me actually think about the problem of testing development tools.

Starting with, “ok, so say Tim Cook or someone in Apple management called me up today and took me up on that tweet but put me in charge of the effort.”   Like, ” Ok big-mouth, here’s a Billion Dollar budget, all the authority you need, a great salary, and relocation, etc for your family, now get down here and lead the effort to fix this!”   What would I do?  (cue deep sinking feeling…  8-O).  But ok, I’m a professional, I’ve been a professional developer, team lead and R&D department builder for software companies for a quarter century, what would I do?

Well, for starters, assume everyone on the development team is very very good and really cares about the project – everyone I’ve interacted with in the Xcode and developer tools team fits that description.  So, it’s not likely to be a “weak programmers” problem.  Process? Complexity? External forces? All candidates for sure.

Ok, let’s think about the situation:

Anyone who’s been in the software development industry for any moderate length of time knows the feeling of being forced to ship code you don’t think is ready because someone besides you sets the schedule (either Marketing, Sales or the realities of the buying season for your market).  It’s not a happy feeling.

In this case the dev tools team is likely pushed/pulled by the hardware release schedule and are probably writing tools targeting hardware that’s half baked and buggy all on it’s own.  Talk about a moving target!  Don’t forget that there are multiple parallel hardware development streams all supported by this same toolset (multiple Macs, multiple iOS device, Apple TV), and multiple OS’s that are also running along depending on these same development tools (Mac OS, iOS, iOS for Apple TV) – and those are just the ones we know about!  Who knows what else is going on in the secret labs?  Rather mind boggling when I spell this out and try to draw a timeline of the various hardware and OS releases in the last 18 months.

Also critical to recognize that Xcode and related tools are a HUGE project.  I don’t have direct knowledge, but it seems likely we’re talking multiple millions of lines of code.  Millions.  If you’ve never worked on a project that large, let me assure you it’s crazy big.  Just finding the code that might be involved in a given bug is a real challenge.  Never mind the edit-compile-test cycle time (!!! see below).   “Just step through the code” means something rather different when there are that many lines of code involved.  I don’t know how many engineers are on the team (or several teams I’d guess), but as you get more people involved things get exponentially more difficult to coordinate and there’s just too much code changing every week to keep up with the whole project.  I’ve never personally been on or led teams larger than 15-20 engineers, but even at that size coordination becomes a challenge all by itself.

How huge? Well I don’t have access to the source for Xcode, but we do have access to llvm & clang so I grabbed those and did a sloccount on it:

SLOC Directory SLOC-by-Language (Sorted)
547769 tools cpp=453850,ansic=56962,objc=27864,python=6437,perl=1973,
374043 lib cpp=368997,ansic=3472,pascal=1542,asm=32
113405 test asm=110511,ml=1173,python=937,ansic=780,sh=4
67753 include cpp=60198,ansic=7555
36745 utils cpp=29453,python=4379,sh=1474,perl=876,ansic=255,
9178 unittests cpp=9178
9080 autoconf sh=9080
6185 examples cpp=4728,ml=1443,ansic=14
4032 bindings ml=2103,ansic=1929
2492 projects sh=2473,ansic=19
446 runtime ansic=446
0 cmake (none)
0 docs (none)
0 top_dir (none)

Totals grouped by language (dominant language first):
cpp: 926404 (79.10%)
asm: 110543 (9.44%)
ansic: 71432 (6.10%)
objc: 27864 (2.38%)
sh: 13127 (1.12%)
python: 11753 (1.00%)
ml: 4719 (0.40%)
perl: 2849 (0.24%)
pascal: 1665 (0.14%)
lisp: 651 (0.06%)
csh: 117 (0.01%)
exp: 4 (0.00%)

So without any of the UI (what we think of as Xcode), or any of the accompanying tools (IB, Instruments, etc) we’re already over a million lines of code.  HUGE. 

To get a sense of the edit-compile-test cycle I foolishly decided to run ‘make’ to see how long it would take to build llvm + clang.  My i7 MBA fans are maxed and the build I started thirty minutes ago just finally finished.  Thirty minutes despite an i7 chip and fast SSD.  And that’s just the compiler pieces, no UI at all, nor the lldb debugger, or the standard libraries, etc etc.

Then we consider just how huge a leap Xcode 4 was from Xcode 3.  It’s easy to think of it as a big stumble because of the problems, but instead think about how much the team added and or changed in the UI? Massive change – tons of new functionality, much of it rather amazing coming from Xcode 3 (really leveraging the new low level compilers).   So, new compilers, debugger, new ObjC runtime enhancements, new technology underneath (really using gcd, blocks, etc etc), and the massive UI rewrite (and it’s a huge multi-app UI when you count all the pieces / tools involved), and new hardware and SDKs to target/support along the way.  This was an incredibly ambitious project on a scale that very few of us Mac or iOS developers has experience with on a regular basis (I have for only one project in 25 years, and so have some of you other developers in the crowd I’m sure, but as a percentage…probably fewer than 10-20% (?) of iOS/Mac developers have dealt with million+ line code bases).

Ok. Now imagine how many bugs there are in the database against this codebase… Especially considering all the duplicates because of the limitations of Radar as it’s currently implemented (my post on that here).   8,810,881 is the first bug number listed in open radar in Jan 2011 and 10,969,283 is the most recent one March 1.  That’s two million bug reports in 14 months, assuming they are consecutive (which seems safe to say),  that’s just over five thousand bug reports every day.  Let that sink in for a minute.  5,000 per day.  wow.  Now most of those are for OS X and iOS presumably, but still.  The magnitude of these projects shouldn’t be underestimated.  Talk about hellish bug scrubs…!

Ok.   So we’ve been reminded of the scope of this project – HUGE! – but how are we going to stabilize it?  Naive answer is, we’ll just hire a bunch of QA and maybe some more developers to help fix bugs!  We have a BILLION DOLLARS, we can afford some more QA!

So, who do you imagine does QA on developer tools? …  Yep, developers do. So, most readers will be familiar with the current iOS/Mac OS X developer market – insanely constrained would be putting it mildly.  In 25 years I’ve seen some crazy stuff, but never offers of $10K for the email address of a qualified developer, payable if they agree to accept a job.  That’s 10K to me just for referring someone who ended up getting hired.  Wild.

Now, as a developer, how excited would you be to write test programs and work with beta developer tools to try and break them, writing up detailed and excellent bug reports for the issues you find… sound like a dream job?  (if so, please contact @Jury asap and I’m sure he’ll put you in touch with the right people! :-)).  I bet QA engineer is probably one of the least appealing Mac/iOS developer jobs most developers can think of.

So, how are we going to convince skilled developers to spend their days messing with flakey broken developer tools and trying to break them in reliable ways and then document this in detail?  We’re going to have to pay them a ton of money – say double the usual bay area salary.  Ouch.   Even with that it still might be difficult to find people who’ll stick around.  At some point they’ll have had all they can take.  Huh, this is becoming harder than we thought.  Ok, I got it.  Unit tests, Automated testing, scripted testing of the UI using UI Automator (cool technology).  Yah, so how excited are you to be writing all that test code? Thought so.  Same problem, different angle.

So I got this far in my thinking and realized this is definitely a non-trivial problem.  Which makes fixing Radar all the more important (see my post on that here).

I’d love to hear from an Apple Developer tools engineer or manager how the Xcode team does handle QA for developer tools.  Now that I’ve tried to think it through, I’m amazed how good Xcode is, considering!

Anyway, that’s 1700 words of meandering to get down some of my thoughts relating to Xcode 4 and stability.  And that’s following 1700 words on Radar.  So I’ve procrastinated enough for one day. 🙂  Maybe you have some suggestions to add in the comments? Go for it.

Thought I’d add some notes on things I’ve been observing via twitter about people having problems with Xcode and then having gotten those problems resolved.  Consider these “tips & things to try if you’re having problems with Xcode SPOD or crashing on you regularly”.

  1. Multiple reports of deleting and re-installing.  I myself found that a clean install made the difference between “unusable” and “nice!”.  For 4.3.1 this was as simple as deleting the in /Applications and downloading a new one from MAS.  Note: having another copy installed on another partition or USB stick or whatever can confuse the installer and it’ll update that one.  The button in MAS should read “Install” if it’s not finding another copy to “Update”.
  2. Deleting derived objects folder (and/or the “build” folder from older projects) can help.  Especially if you were using an older version of Xcode 4 which may have left less then pristine indexing (or whatever) files.  Xcode Organizer window, Projects tab, …   This has fixed the continually re-indexing problem that some have had.
  3. Look to see if your headers are including an individual file from a framework instead of the main framework header file (e.g., #import <QuartzCore/CoreAnimation.h> instead of #import <QuartzCore/QuartzCore.h> (99% sure this is what @Jury was talking about).  And yah, I agree, this shouldn’t be a problem, but I’ve seen @Jury talk about how it might be and at least one developer then reported that changing this fixed his problems.

Any other tips that worked to make Xcode stable for you?  Drop them in the comments and share the knowledge! Thanks.

P.S. I’ve recently posted a few Xcode 4 tips that you might find useful:

And I have at least one more I need to write up.

A dev who’s making his living using Xcode and who is experiencing problems with Xcode 4 hanging every hour replied to my suggestion to “Sampling Xcode during the hang might help you figure out what’s wrong, and would be good fodder for a radar” with:

Nope. Too busy to debug Xcode for Apple 😦

While I can understand the sentiment (I don’t get paid to track down bugs in Xcode either!), I also think this is a bit short sighted and ignores a pretty important fact:  Consider just how extensive the tools we get from Apple for free are.  I mean, really:  Instruments alone is pretty amazing, not to mention the OpenGL Profiler, etc etc.  And Xcode 4, for all it’s rough edges, is a pretty amazingly powerful IDE.  And before you chant “open source tools!”, have you actually used any? Like, say Eclipse? ugh.  So, I think it’s worth considering if spending 1 hour a week to help improve the tools you use every day to make your living is a small price to pay for said tools or not.  Even at my hourly rates, I think the answer is: this is a great deal.  Give it a ponder and then do what feels right to you.

Xcode 4.x tip: option key directs open action to Assistant editor + direct open

March 11, 2012

This just came up on twitter so I thought I’d mention it with a few more characters available:  holding down the option key whilst performing an action that will open/show a view in the editor will put that view into the Assistant editor.  Examples:

  • option + click on a file in the Project Navigator
  • option + select the “Open” button in the File Open dialog
  • option + select the “Open” button in the Open Quickly dialog (Cmd-shift+O to get this dialog).
  • option + cmd + click on any clickable symbol in source view (method, class, file name, instance var name, most type names, etc)
  • option + click on the .h file listed in the documentation pop-over (option click on a symbol in editor to get this).
  • option + click on a file listed in Search navigator, Breakpoints navigator, Log navigator, etc.
  • option + select a file you navigated to in the jump bar of the main editor.

and so on.  You get the idea.  Very nice Xcode 4 feature.

OOOOH, bonus tip(s)!

  1. You can have more than one assistant editor.  The “+” at the top right corner of the Assistant editor adds another.
  2. when you use the option + action feature above, add a shift key for real magic – direct where you want the item to open, from current main editor to existing Assistant editor, to a new Assistant editor, to a new window.   This is made available via a small little navigation pop-up HUD style window like this:

Note that you can navigate around in this pop-up HUD with the arrow keys, and command+left/right arrow to move between the windows.  very nice!

Xcode 4.x tip – how to get a separate working debugger window

March 6, 2012

Just figured this out today after trying once before.  If you have a multi-monitor setup like I do you may like to have the console & debugger be in a separate window.  Xcode 4 moved to a predominately single window interface which is a bummer for multi-monitor setups.  I tried opening a new window and collapsing all but the debugger / console panes but the console wouldn’t take commands and was effectively useless..

Turns out there is a trick menu item:  View -> Debug Area -> Activate Console 

So with the cursor in the console in the second window, select the above mentioned menu item and Bingo! a working debug / console view in a separate window. sweet!


Now to file an enhancement request for a behavior to open a separate window and configure it like this automatically 🙂

The psychology of Radar is broken

March 6, 2012

Xcode 4 has been a mixed blessing for quite a few Mac OS X & iOS developers and recently Daniel Pasco posted a good blurb asking people to stop whining and file bug reports already and why.  It’s a good post.  I’ve actually thought about Xcode 4 and the challenge of producing an IDE/Dev tools in the current climate and I need to write a post on that.  I’ve worked on an IDE way back when, and also on at least one very large project – so I know some of the challenges of both.  However, this post is going to be about Radar because if Radar worked better we outside developers could support Apple’s Xcode & Developer Tools team better and that would be good for all of us.

Radar has a lot of problems and both Daniel and now Martin Pilkington in his recent post have filed bug reports and made suggestions to fix Radar.  I think they both make good points, but I don’t think their general “make radar searchable by 3rd party devs” is going to fly.  I’ve had this conversation with Apple people over the years and some have been “yah, I wish!  I’ve tried, they’ll never go for that…”  Further, I’ve certainly had clients for whom completely readable bug reports would have been a problem.

First, let’s take a quick look at what’s wrong with Radar as is:  to me the biggest issue is that there is a strong Negative Reinforcement dynamic with the current system.  Negative reinforcement is a strong behavioral modifier and is used to decrease the occurrences of a behavior (think punishing a misbehaving child or pet).  As such, it’s a serious problem that Radar provides continual negative reinforcement for filing bug reports.  Every “duplicate” email I get is a strong negative reinforcement for having wasted the time making a good bug report:  I always think and/or feel “All that work and it was already in their system!  DARN!  WHAT A WASTE!”  ok, that’s the PG version :).  I know, I know, it’s not a total waste because Apple uses the duplicate filing as some kind of primitive “voting system”, but that’s too sub-par to feel valuable.

I think both Daniel & Martin are identifying this same dynamic when they point to lack of searchability as the key problem to fix in Radar.

This “file a duplicate to vote up the bug” seems like a terrible idea to me and  here’s why:  not only does it create the negative reinforcement dynamic, it also creates a ton of unnecessary work for both Apple staff and outside developers.  On Apple’s side someone (many people I’d imagine) has to go through all the filed bug reports doing presumably manual searches to try and determine if a newly filed bug report is a duplicate or not.  (Hey Apple! We developers would be willing, no, we want to do those searches for you, for free! ) 

But the bulk of the extra work is shouldered by the outside developers who are writing the unnecessary duplicate bug reports.

Like any developer I hate receiving a badly written bug report.  As a result, it’s difficult for me to file a bug report that hasn’t been made reproducible, doesn’t have the simplest sample code necessary to demonstrate the bug, and lacks a good description of the circumstances under which it does and doesn’t happen.  (The only exception is a bug that I have a crash log for but can’t reproduce; these I’ll write up a bug report if I can add value to the crash report indicating what I was doing and what I saw.  But I’ll still spend time trying to make it reproducible).  For me this means that writing up a “proper” bug report takes between 1 and 4 hours (sometimes more).

Now it seems to me that Apple would really rather have their system full of well written bug reports and not clogged up with a bunch of badly written incomplete bug reports.  So beyond saving them staff time, they’ll also end up with higher quality bug reports (we’re more motivated, or less de-motivated more likely).

Now I’ve gone through phases of bug report writing.  Sometimes I write a lot, but generally I find it too depressing to get the “duplicate!” emails and so I don’t do very many.  I was wondering if I was exaggerating the strength of the negative reinforcement embodied by Radar as is, so I went back and looked through the bug reports I’ve filed with Radar in the last 10 years (apparently I didn’t use Radar before that or they were deleted…?).  Roughly speaking, 90% of the bug reports I filed were marked duplicates, 5% were marked as closed (good bugs, worth filing) and 5% were closed as ‘not going to fix’ or ‘as designed’ or something else I mostly disagreed with :).

This means that for every 20 bug reports I file in Radar I’m only filing 1 that is a “keeper” (not a duplicate and something that gets fixed).  Since each bug report takes 1-4 hours this means that for every “good” bug report I file I’ve spent somewhere between 20 and 80 hours to get that one good bug report.  Yikes!  Let’s call it 50 hours average.  Now most Mac/iOS developers (in the USA anyway) make something like $50+/hr salary (which costs their employer roughly $100/hr with taxes, benefits, etc) or $100+/hr contract rate.  This means that one good bug report effectively costs $5000 or more to file, on average.

Now I work primarily as a contractor for software publishers or hardware manufacturers and for them this is not a cost they want to absorb and here’s why:  even if the bug gets fixed it won’t get fixed immediately and that means the client also has to pay to have a work-around figured out and coded up no matter what (and this isn’t always trivial or even possible).  So they end up paying twice.  Ouch!

So we have a second strong negative reinforcement for filing bug reports – someone has to pay:  either the client pays $ and it impacts the schedule or it comes out of the developer’s personal time.  Bummer.  Bigger bummer when 95% of them don’t end up as new useful bug reports.  *Very* demotivating, to say the least.

Now some people are copying bug reports to open radar and that’s very generous of them, but honestly it’s hard not to feel like this is a completely unnecessary waste of time because of a system that already takes too much time.

Ok.  So, using Radar as it’s currently implemented is clearly a bummer (to me and many others if twitter and how few of my colleagues write bugs is any indication).  I’m not just writing to whine, so what’s the fix?  Daniel created four bug reports on radar itself which he posted on open radar for all to read and duplicate (thanks, but “ugh” – see above :)).  See his post for details.  Martin also created some bug reports as referenced in his article.

I totally agree we as developers need the ability to search Radar for existing bug reports.  However, I don’t think the proposal as written will fly and here’s why:  Look at all the leaks on MacRumors and such about stuff that’s covered under developer NDA.  Sad to say it, but there are dishonest developers among us (I wish they’d stop – the result of their blabbing can’t be anything other than Apple sharing less and less with developers which is definitely bad for us).  So my proposal for Radar goes like this:

  1. Make the summary of all 3rd party bug reports searchable by logged in radar users.
  2. Make the balance of the bug report default to not-readable/not-searchable but have a button that lets a developer mark a bug report as viewable by all logged in radar users.  This let’s the person authoring the bug indicate the level of secrecy they need.  They are the only ones who know, so this makes sense.
  3. Allow other registered developers to “vote up” a bug to indicate that a given bug is also a problem for them.
  4. Allow developers to add their email address (or their radar account) to an existing bug report to be automatically kept informed about changes to the bug (when it’s fixed, new info added, etc).
  5. Allow developers & DTS or whomever to add notes on work-arounds for bugs.
  6. When the bug is fixed have Apple engineering mark what version of the relevant OS or component the fix is in (this might involve a slight change of workflow at Apple).
  7. Allow other registered developers to attach additional crash reports, sample code, etc to any existing bug report to aid Apple engineering in finding and fixing the root problem.
  8. I also like Martin’s “ability to create drafts of bug reports” suggestion.  I’ve been bitten by Radar’s “session timeout” problem enough times (and ended up skipped complex bug reports as a result) that this would be welcome.

So, what’s the result?  Well, developers are motivated to file quality bug reports because they’ll know when they “have a live one” on their hands.  So Radar will have more high quality bug reports, fewer duplicates (nearly none), and there will be more information about each bug that IS in Radar.  All this will result in Apple being able to devote fewer staff to finding duplicates and being more efficient and effective and managing and fixing bugs. And that means less buggy software shipping to customers and developers.

I think everyone wins and I most developers I’ve spoken with agree that this is where the majority of the frustration comes from – if everyone wins, why hasn’t Apple fixed this yet?  I can only conclude that we are missing some information and/or there is some lawyer at Apple that’s blocking it.  I’ve suggested to Apple people that we get a bunch of long-time Mac/iOS developers together with the appropriate Apple people and have a working conversation about this.  Developers are, by and large, intelligent and capable problem solvers, why not take advantage of that and the power of a broad spectrum of experience and backgrounds to solve this problem already?!  I’ve never gotten an answer, but I’m intentionally not “high profile” so I guess that’s not entirely surprising.

My bug report referencing this blog post is: rdar://10995739  so feel free to duplicate it.  Here’s what it says:

Radar has some serious issues which I’ve outlined here:
along with some constructive suggestions for improving it.



Sorry about the broken posts in the RSS feed – wordpress bug

March 2, 2012

Just a quick apology for the multiple broken post links in the RSS feed. WordPress’ new “home page quick entry post form” was buggy and was posting drafts (!). So I deleted those drafts but apparently they’re still listed in the RSS feed but give a 404 error when selected.

The real posts should show up by monday or so; waiting on a couple of friends to review them for idiocy, er, errors 😛