Sunday, August 3, 2014

GDG Boulder

I moved from California to Colorado this May, and want to get more involved with the local tech community. As an Android developer, especially as a Google Developers Expert, my natural choice is to help organize GDG Boulder.

Last Thursday we met for Lightning talks and I/O Show and Tell.

We had two speakers for lightning talks:


Ian Douglas on SendGrid's partnership with Google.


Robert Kluin on Go.

After that, the floor was open for I/O Show and Tell. People who went to Google I/O brought their Android Wear watches, Cardboard, Android TV and devices with Android L preview. Everybody got to play with the new toys and share their thoughts. It was great fun.


Immersed in Cardboard

Next, on August 21, our meetup is going to feature mind-controlled robots. Join us!

Tuesday, June 24, 2014

GetSET App Inventor Workshop

I have always wanted to try App Inventor, so when my friend asked me if I'd be interested to teach a workshop on mobile programming for GetSET in Santa Clara, I immediately said yes. But thing got a bit more complicated when I decided to move to Colorado. I wasn't sure I would be able to run the workshop until I gave a talk at Mills College and met Renee there, who volunteered to be my TA.

Preparation

There were a few things we need to do to prepare for the workshop:

  1. Come up with a lesson plan
  2. Make sure we have Android devices
  3. Set up the workstations in the computer lab

Renee helped with all three. She was in Prof Ellen Spertus' class, who wrote a book on App Inventor. Ellen said that a good workshop format is demo-walkthough-explore, meaning that I would first code up an app in front of the whole class, explaning basic concepts. Then I would talk through a simple tutorial, with everyone following on their own computer. Finally it would be free time. The students could build whatever app they want. I thought this was a great format, getting more and more hands-on.

For the devices, Ellen lent me a few Nexus 7 tablets for the workshop, and Renee took care of the pick up and drop off. Finally, the compuer lab setup, which was the most complicated part. The workstations in the computer lab did not have wifi, so we had to use USB connection. GetSET was hosted by Santa Clara University, and only admins can install software. They asked me to send the list of software to be installed, but we needed to verify that the workstation could connect to the tablet after installation, so this could not be done remotely. Fortunaely Renee was able to drive down to Santa Clara for the initial installation. At that point we did not have access to the Mills tablets yet, so I borrowed a Nexus 7 from a friend to leave with the lab technician to repeat the installation across all the lab computers.

Workshop

Yesterday was the big day. The workshop was 9am to noon, and Renee and I showed up at 8am to get everything ready, with two more volunteers. To our horror, the tablets were not connecting to the workstations. We had a few theories: the driver was installed under admin mode, but we ran the workshop in user mode. The Nexus 7 for the initial set up was first generation, but the Mills tablets were newer. In any case, we could not use the workstations. We ended up using laptops from the volunteers, which meant 4 computers, 10 students. Not too bad a ratio.

With all that taken care of, we were ready to make apps! I demoed a raffle app that say the name of the winner:

Then we walked through Hello Purr together. There was much laughter as the tablets meowed.

Finally it was free time. The girls really enjoyed making apps, and did not want to leave when I announced a bathroom break in the middle! Different teams made different apps. Some added doodling capabilities to the cat, some made whack-a-mole, some made a pirate ship go get gold, some made text-to-speech for Spanish. It was great fun.

Next steps

I really enjoyed running this workshop. App Inventor is a great way to introduce basic programming concepts because it is lightweight to set up, and running apps on the kids' phones made it much more personal. I am not sure how to graduate them from App Inventor to Android, though. App Inventor only has a subset of the Android capabilities, and right now there is no way to convert an App Inventor project to Java+XML for Android, so they would have to rewrite the whole app if they wanted to step beyond the App Inventor functionalities.

Have you taught App Inventor before? What do you do when your students want to step up the game?

Adventures of the Mind

Public speaking has led to my different adventures for me. I gave an Ignite talk on hackathons at Google I/O 2012, sharing the stage with my former boss Peter Norvig. He was mentoring at Adventures of the Mind two years ago, which expanded to include a hackathon this year. He introduced me to the organizers to help run the hackathon, and that was how I got to mentor almost 200 brilliant high school students last week.

Hackathon


Kick off

I ran the hackathon with Mayank Jain, who has been organizing hackathons for high school students under Pilot. The Adventures of the Mind edition followed the same format as Pilot. Students can choose whatever language and platform they want, and go through with team formation, workshops, coding and pitching in 24 hours. I was not sure if that would work, especially since most of the students had no coding experience, and we only had 10 hours. Mayank assured me that previous events went quite well, and students were able to achieve quite a bit in a short amount of time. So we kept the Pilot format.

We started the hackathon with workshops. Cheston from mashery led an HTML5 one, and I led an Android one. We emailed instructions ahead of time asking the students to set up Xcode for iOS, Android Studio for Android, or a text editor for html/javascript on that laptop, and go through some tutorials. But naturally some people did not get a chance to do it, others struggled with installation problems. Many people had Java problems on their Windows computers, but I wasn't able to help them since I don't have Windows, and was not familiar with the failure cases.

After a while I made a radical decision: I switched to App Inventor. I was going to teach App Inventor the week after Adventures of the Mind anyway, so I already had a lesson plan. That went much more smoothly since the setup was much simpler: no install, no drivers. With that, the students were ready to write their own apps.

During the day the mentors went around to help the students, but we did not have enough mentors: only 5 mentors, and almost 200 kids. A batch of mentors dropped out last minute, and it was very difficult to find replacement on such short notice, especially since the hackathon was on a Tuesday. As a result I was only able to help a few teams, and felt very bad about the other teams that I could not help.

Despite all the struggles, the students really wowed us during the presentation. Here is a sample of their apps:


#feedtheworld: a website to direct recycling money to charities

Mind Scan: barcode scanner to exchange contact information

TravelPilot: recommend an itinerary for any destination

Overall the hackathon was rather well. A lot of students came up with wonderful ideas, learned to code, and demoed their app. But some teams could not finish their app because of technical hurdles, not knowing if their idea would be complicated to implement, and lack of guidance in general. If I were to do this again I would make sure we have a lot more mentors.

Sessions

The hackathon was just one part of this week-long program. There was a lot of interesting sessions. Here are some highlights:


Danny Oppenheimer on perception

Jini Kim on fixing healthcare.gov

Jennifer Shahade playing chess against 15 students at the same time

Nancy Segal and her twin research

Outings

We also got to visit quite a few interesting places in Los Angeles:


Autry Museum

LA Zoo

Gamble House. I love all the different wood texture.

JPL: we got to see the mission control room!

My first summer camp

It was really cool to hang out with all these smart kids, visit interesting places, and also get to know the other mentors. Come to think of it, this is actually my very first summer camp! I made many new friends, just like the students :)


With Nancy Segal

With Shahara Ahmad-Llewellyn

With Amy Tan, her husband, and their dogs.

Monday, June 2, 2014

NavigationDrawer creates fragment twice on rotation

I have an app with a navigation drawer. Everything was peachy until I rotated the screen, and noticed that the views are not preserving their states. Digging deeper I realized onCreateView for my fragment was called twice, the second time without saved state. What did I do wrong?

Logging

To isolate the problem, I created another app, generated the activity from the Android Studio template, and just added a single logging line:

@Override
public View onCreateView(LayoutInflater inflater, 
    ViewGroup container, Bundle savedInstanceState) {
  Log.i("sqisland", "onCreateView. Saved state? " 
    + (savedInstanceState != null));
  View rootView = inflater.inflate(
    R.layout.fragment_main, container, false);
  return rootView;
}

And indeed, onCreateView called twice on rotation, once with saved state and once without:

I/sqisland(24201): onCreateView. Saved state? true
I/sqisland(24201): onCreateView. Saved state? false

More logging

I enabled debugging on the FragmentManager to get more information:

FragmentManager.enableDebugLogging(true);

Time to read the tea leaves, and there are lots of it. I picked out the relevant part:

Saved state of NavigationDrawerFragment
               {41ea1a38 #0 id=0x7f07003e}
Saved state of PlaceholderFragment
               {41f027a8 #1 id=0x7f07003d}
Freeing fragment index NavigationDrawerFragment
                       {41ea1a38 #0 id=0x7f07003e}
Freeing fragment index PlaceholderFragment
                       {41f027a8 #1 id=0x7f07003d}

On rotation, the FragmentManager saves the states of the NavigationDrawerFragment and PlaceholderFragment, then frees them.

Instantiated fragment NavigationDrawerFragment
                     {41f90658 #0 id=0x7f07003e}
restoreAllState: active #0: NavigationDrawerFragment
                            {41f90658 #0 id=0x7f07003e}
Instantiated fragment PlaceholderFragment
                      {41f91018 #1 id=0x7f07003d}
restoreAllState: active #1: PlaceholderFragment
                            {41f91018 #1 id=0x7f07003d}

Then it instantiates new fragments for the new activity, and restores the saved states.

moveto CREATED: PlaceholderFragment
                {41f91018 #1 id=0x7f07003d}
moveto CREATED: NavigationDrawerFragment
                {41f90658 #0 id=0x7f07003e}

Commit: BackStackEntry{41ed3088}
   mName=null mIndex=-1 mCommitted=false
   Operations:
     Op #0: REPLACE PlaceholderFragment
                    {41f849c8 id=0x7f07003d}

Next it moves the fragments to onCreate. There is a commit because onCreate in NavigationDrawerFragment calls selectItem, which triggers the activity to instantiate a new PlaceholderFragment.

@Override
public void onNavigationDrawerItemSelected(int position) {
  // update the main content by replacing fragments
  FragmentManager fragmentManager = getSupportFragmentManager();
  fragmentManager.beginTransaction()
      .replace(R.id.container, 
               PlaceholderFragment.newInstance(position + 1))
      .commit();
}
Run: BackStackEntry{41ed3088}
OP_REPLACE: adding=PlaceholderFragment
                   {41f849c8 id=0x7f07003d} 
            old=NavigationDrawerFragment
                {41f90658 #0 id=0x7f07003e}

Now the FragmentManager runs the transcation, looking for fragment id 0x7f07003d for replacement, which is R.id.container. It skipped the NavigationDrawerFragment because the id does not match.

OP_REPLACE: adding=PlaceholderFragment
                   {41f849c8 id=0x7f07003d} 
            old=PlaceholderFragment
                {41f91018 #1 id=0x7f07003d}

Next it found the PlaceholderFragment which it restored from rotation, with id 0x7f07003d. Bingo! This is the fragment to replace.

remove: PlaceholderFragment
        {41f91018 #1 id=0x7f07003d} nesting=0
Freeing fragment index PlaceholderFragment
                      {41f91018 #1 id=0x7f07003d}
add: PlaceholderFragment
     {41f849c8 id=0x7f07003d}
Allocated fragment index PlaceholderFragment
                         {41f849c8 #1 id=0x7f07003d}
moveto CREATED: PlaceholderFragment
                {41f849c8 #1 id=0x7f07003d}

Replacing a fragment means removing it and adding it back. Afterwards, the FragmentManager moves the fragment into onCreate. This is why I see onCreate called twice, once with saved state, and once without.

The fix

A quick summary:

  1. Device was rotated. FragmentManager saves state for all fragments, then frees them.
  2. New activity. New fragments get instantiated, with states restored.
  3. Fragments get moved into onCreate.
  4. onCreate of NavigationDrawerFragment triggers a replace transaction.
  5. The replace transaction removes the restored PlaceholderFragment with a new one, which has no saved state.

To prevent the second PlaceholderFragment from being created, we need to tell NavigationDrawerFragment not to trigger a replace transaction when it is restored from saved state. I added a parameter to selectItem:

private void selectItem(
    int position, boolean fromSavedInstanceState) {
  mCurrentSelectedPosition = position;
  if (mDrawerListView != null) {
    mDrawerListView.setItemChecked(position, true);
  }
  if (mDrawerLayout != null) {
    mDrawerLayout.closeDrawer(mFragmentContainerView);
  }
  if (mCallbacks != null) {
    mCallbacks.onNavigationDrawerItemSelected(
      position, fromSavedInstanceState);
  }
}

When the selectItem is called from onCreate, fromSavedInstanceState = (savedInstanceState != null). When it is called from the ListView OnItemClickListener, it is false.

The activity only commit the replace transaction if needed:

@Override
public void onNavigationDrawerItemSelected(
    int position, boolean fromSavedInstanceState) {
  if (!fromSavedInstanceState) {
    // update the main content by replacing fragments
    FragmentManager fragmentManager = getSupportFragmentManager();
    fragmentManager.beginTransaction()
        .replace(R.id.container, 
                 PlaceholderFragment.newInstance(position + 1))
        .commit();
    }
}

Don't forget to change the signature of the interface:

public static interface NavigationDrawerCallbacks {
  void onNavigationDrawerItemSelected(
    int position, boolean fromSavedInstanceState);
}

Done, right? Not so fast. This indeed removed the second creation of PlaceholderFragment, but the action bar title is not updated. Here is why:

@Override
public void onAttach(Activity activity) {
  super.onAttach(activity);
  ((MainActivity) activity).onSectionAttached(
      getArguments().getInt(ARG_SECTION_NUMBER));
}

We are updating the title onAttach, but when the fragment is restored by the FragmentManager, it is attached before activity onCreate is called, so the title gets overwritten by the app title. Instead, we want to update the title after activity onCreate is called.

Time to consult the fragment lifecycle:

  1. onAttach(Activity) called once the fragment is associated with its activity.
  2. onCreate(Bundle) called to do initial creation of the fragment.
  3. onCreateView(LayoutInflater, ViewGroup, Bundle) creates and returns the view hierarchy associated with the fragment.
  4. onActivityCreated(Bundle) tells the fragment that its activity has completed its own Activity.onCreate().
  5. onViewStateRestored(Bundle) tells the fragment that all of the saved state of its view hierarchy has been restored.
  6. onStart() makes the fragment visible to the user (based on its containing activity being started).
  7. onResume() makes the fragment interacting with the user (based on its containing activity being resumed).

onActivityCreated it is:

@Override
public void onActivityCreated(Bundle savedInstanceState) {
  super.onActivityCreated(savedInstanceState);
  ((MainActivity) getActivity()).updateTitle(
    getArguments().getInt(ARG_SECTION_NUMBER));
}

I renamed onSectionAttached to updateTitle as well:

public void updateTitle(int number) {
  switch (number) {
    case 1:
      mTitle = getString(R.string.title_section1);
      break;
    case 2:
      mTitle = getString(R.string.title_section2);
      break;
    case 3:
      mTitle = getString(R.string.title_section3);
      break;
  }
}

Yay, everything works!

Did you read all the tea leaves with me? Wow. Go get yourself a nice drink. You deserve it.

Friday, May 16, 2014

Ignite Boulder

I love Ignite - 5 minute talk, 20 auto-advancing slides. The format is challenging, forcing you to think hard about what you want to say, and timing it just right. The resulting talks are fast-paced, polished, and lots of fun to watch. I just moved from California to Colorado, and as luck would have it Ignite Boulder was happening 11 days after my arrival. What a perfect way to get to know the community!

Ignite in disguise

The evening opened with an introduction from Andrew Hyde:

He welcomed us, introduced us to Ignite, and gave us the usual low-down. I didn't think much about it until a minute or so later, when I realized that the slides in the background were changing subtly, highlighting a different line... every 15 seconds! He was giving an Ignite talk! It was customary to open an Ignite event with an Ignite talk, but this was so sneaky! I loved it.

Funny, inspiring, and making a difference

During the course of the evening the speakers serenaded us with many interesting topics:






The Boulder community

This slide from Becca Close summarizes the Boulder community very well:

Boulder is a hyper-connected place, where everybody knows everybody. While such a tight-knit community makes the dating scene awkward (Becca claims that you are practically relatives with your date), it also brings people together. I love how friendly the Boulderites are, and how they care about each other. Very excited to be a part of it!

Saturday, April 19, 2014

Monkey Write presentation

When I was a kid, I learned Chinese the old fashion way. Each day we had to practice one character, and fill it in a 10 by 10 grid. It was boring and repetitive. With Monkey Write I want to make it fun to learn to write, and I always had children in mind. However, as a one-woman development shop there were always so many things to do, and I did not get a chance to cater the app to schools.

That is, until I heard from a parent today. Lorie emailed me and let me know that her son really enjoys Monkey Write, and she wondered if it would be possible to have workbooks that align with the syllabus at his Chinese immersion school. This is the birth of the Immersion series in Monkey Write. I have since published multiple workbooks under the Immersion series. Other parents got curious about Monkey Write, so Lorie gave a presentation on it!

This is the most rewarding part of being an independent developer. I get to talk directly with my users, hear their stories, and add features that they care about. Thank you for making my day, Lorie!

Tuesday, January 28, 2014

Monkey Write: Brush strokes

Back when I started Monkey Write at the AT&T hackathon, I was already varying the pen stroke width by the pressure of the touch events. But the pressure ranges change a lot on depending on the device. I was not able to get consistent stroke rendering with all those pressure ranges, so I only vary the stroke width by pressure if I know it is from the active stylus reported via an HTC PenEvent.

I was really excited when I saw the beautiful Markers app. Its pressure-sensitive strokes work on many devices, plus it is open sourced with Apache License 2.0, so all I have to do is to integrate that into Monkey Write. Well, all I have to do is to find time to integrate that into Monkey Write, which I finally did!

PressureCooker

When I first looked into the Markers code I was very amused by the PressureCooker class. What a name! It calibrates the pressure coming from a series of touch events, which is what I need. I lifted that class and put it into Monkey Write, but that alone did not make beautiful strokes. This was where I stopped the first time I looked into Markers, for the rest involves a more complicated co-ordination among the Slate, TiledBitmapCanvas and SpotFilter classes.

Markers architecture

I finally set aside some time to understand how Markers work.

  1. Convert each incoming touch event into a Spot.
  2. Add the Spot to a SpotFilter.
  3. The SpotFilter takes a Plotter in its constructor. After filtering it calls the plot() function of the Plotter.
  4. The Plotter renders the Spot on screen. In Markers this is handled by the inner class MarkersPlotter in Slate, which draws on the TiledBitmapCanvas.

Where is the PressureCooker, you ask? It is used inside the plot() function. Instead of using Spot.pressure directly, it is calibrated by the PressureCooker.

Monkey Write modifications

Here is what I did to incorporate Markers stroke rendering into Monkey Write:

  1. Replace my own class with Spot to store touch events.
  2. Add a TiledBitmapCanvas to the character writing custom view (called SketchPad).
  3. Make SketchPad implement Plotter, which takes a Spot and renders to the TiledBitmapCanvas.
  4. Add a SpotFilter to SketchPad. As touch events are captured by onTouch, pass the Spots to the SpotFilter.
  5. In SketchPad.onDraw(), call TiledBitmapCanvas.drawTo() after rendering the base character to show the pen strokes.
  6. After the user writes a stroke, grade it. If it was not a good stroke, call TiledBitmapCanvas.step(-1) to remove that stroke.

Pen styles

After I set up the basic pressure cooking and spot filtering I started to experiment with different pen styles. This essentially means changing the pen tip, or how to render each touch point aka Spot.

Basic style

The basic style renders the pen tip as a solid circle. Fairly straight forward:

c.drawCircle(x, y, r, mPaint);

Brush style

Markers has an airbrush style, which draws a bitmap as the pen tip. I looked at the bitmap and thought, hey, that's just a RadialGradient! I decided to generate that programmatically so I can vary the alpha value on the fly:

private Shader createBrushShader(
    float width, float alphaStart, float alphaEnd) {
  final float center = width / 2;
  final float radius = Math.max(1, width / 2);
  final int red = Color.red(mPaint.getColor());
  final int green = Color.green(mPaint.getColor());
  final int blue = Color.blue(mPaint.getColor());
  return new RadialGradient(
      center, center, radius,
      Color.argb(alphaStart, red, green, blue),
      Color.argb(alphaEnd, red, green, blue),
      Shader.TileMode.CLAMP);
}
mPaint.setShader(mBrushShader);
c.drawCircle(x, y, r, mPaint);
I update mBrushShader with createBrushShader() whenever the user changes the width or alpha from the UI.

Pencil style

Once I started playing with Shaders I could not stop. I decided to mimic a pencil stroke on a rough paper by plotting little dots at the pen tip.

private Shader createPencilShader(float alpha) {
  final int size = 32;
  int color = Color.rgb(
      Color.red(mPaint.getColor()),
      Color.green(mPaint.getColor()),
      Color.blue(mPaint.getColor()));
  float threshold = alpha / 255f;

  int[] colors = new int[size * size];
  for (int i = 0; i < colors.length; ++i) {
    colors[i] = (Math.random() >= threshold) ? 0 : color;
  }

  Bitmap bitmap = Bitmap.createBitmap(
      colors, size, size, Bitmap.Config.ARGB_8888);
  return new BitmapShader(
      bitmap, TileMode.REPEAT, TileMode.REPEAT);
}
mPaint.setShader(mPencilShader);
c.drawCircle(x, y, r, mPaint);

I use a BitmapShader to draw the little dots. It is a tiling bitmap, each pixel is either transparent or the chosen color. The lower the alpha value, the more transparent pixels.

Constant width

Finally I want to provide an option for users who don't want variable width. This is achieved by ignoring the pressure from the touch event and supplying a constant value to the pen tip renderer.

Mix and match

With that you can have a lot of fun making different pen styles.

Here is a very transparent blue stroke. Looks like water, doesn't it?

You can pick different styles for different strokes:

Hopefully these beautiful strokes will make it even more fun to practice writing Chinese. Download Monkey Write and try them out!