Firebug is a web developer's best friend. Without it, there are times when we'd be absolutely lost. Luckily it is actively developed and well-supported.

The 1.1 beta version was released in the middle of February, but I'm just getting around to upgrading to it. Until I started to actively use Firefox 3, I didn't see much of a need to upgrade (Firebug 1.1 can be installed on Firefox 3), but boy was I wrong. There are a couple of big additions that make this a must-have upgrade for me:

• eval() debugging - this is the big one. I can't tell you how many times I've run into cryptic issues using the eval() statement. Firebug 1.1 handles this by displaying code sent to eval() in the Script panel as a new source file. This code can then be debugged just like any other JavaScript in Firebug. If you don't want to debug eval() code, you can always turn this option off. You'll get better performance with it turned off.
• Aptana IDE editor integration - I use Aptana's Eclipse plugin for all of my CSS, HTML, and JavaScript coding, and integration between Aptana and Firebug only makes my life that much easier.
• Cache tab for Net panel - this should make it easy to verify if a resource is cached or not.
• Supports Firefox 3 - already mentioned this one above, but thought it should be mentioned again.

Jason Levitt recently announced in a Yahoo! Developer Network blog entry (dated December 21, 2007) that Yahoo! Maps is now pure JavaScript. In the past it was a hybrid Flash/JavaScript application. The new port brings, according to Levitt, "at least double the performance of the previous Flash-based version". And of note to developers, the enhanced version of the Yahoo! Maps AJAX API which is being used in the newest version of Yahoo! Maps will be made available to developers sometime in 2008.

To be honest, I haven't paid all that much attention to Yahoo! Maps over the last couple of years, as both their "flagship" application (maps.yahoo.com) and their API have, to be quite honest, failed to impress. And (staying honest here), after using the application and perusing the API again tonight, I'll have to say that I'm still not all that impressed:

• The application's responsiveness just doesn't seem to be up to par when compared to Google and Microsoft's mapping applications (maps.google.com and maps.live.com, respectively). Please note that at this point, this is just an observation; I haven't conducted - or even looked for - any performance tests, but I definitely think that Yahoo! has a long way to go before they'll be able to start seriously competing against Google and Microsoft.
• On top of this, the look and feel of the application just doesn't do it for me. In my opinion, Microsoft - with its latest release - has the best looking and most usable interface out there. Google still has more "Wow!" features ("Wow! I can drag a route to change it!" ), but I don't think that I've ever actually used any of them.

To close, I'm disappointed that Yahoo! - one of the companies that really led the way in the early (2005) web map API days - has fallen so far behind. I hope that this coming version of their Maps AJAX API will bring them back into the competition.

UPDATE: You can now download this blog post.

My apologies for the super-long post; I was originally going to break this up into multiple posts, but my week at the conference filled up very quickly and I couldn't find time to write out all my thoughts and experiences and post them daily. To try and facilitate reading of this post, I have, however, broken it up by day and by technical workshop. And at the end of the post I have written about some miscellaneous tidbits that I discovered over the course of the week. I expect that in the coming weeks I'll be writing more in-depth about many of the topics that I am able to cover only briefly below, as many of the developments have major ramifications for anyone who works with ESRI software.

Judging by a recent post from an austere colleague, I guess that being a participant at the ESRI User Conference (from here on referred to as the UC) puts me (and many others - you know who you are) in a position that brings an inherent level of responsibility. I'll be honest, though, I wasn't aware of these overarching responsibilities when I signed up; my apologies for not reporting the juicy details sooner. Fortunately, though, my colleagues seem to have adequately filled the void that I unknowingly contributed to over this past week.

That said, I walk out of San Diego a week older and with a number of news items, tidbits, and rumors - enough, at least, to get me pretty excited. I'm not sure that there were any monumental announcements or "WOW!" moments this year (a lot of the big announcements were spoiled by Jack's answers to the Questions and Answers that were released the week before the UC), but ESRI and some of their partners talked about coming developments that will definitely push GIS in a new direction over the upcoming year, for good or for worse.

Following are my own personal highlights from this year's UC. Please note that I am a .NET developer and self-proclaimed technologist, and am, therefore, totally geeky and focused on topics that are of interest to me. I didn't get much out of the "Everyone Loves/Needs/Uses GIS" sessions, the plenary talks, or the desktop productivity sessions, although overall it was nice, as a colleague of mine said, to hear others talking about GIS when having dinner at a random restaurant in the Gas Lamp District. I did, however, get a lot out of interacting with ESRI developers and the various vendors, and these are the moments  that I want to pass on in this write-up.

Monday, June 18th

Plenary

A good portion of the plenary fit into the "Everyone Loves/Needs/Uses GIS" category mentioned above. I'm glad to know how GIS is being used around the world, but as a GIS professional I don't have to be sold on the idea of GIS as an invaluable tool. The demonstrations and announcements at the end of the session caught most of the news, although unfortunately a lot of them fit into enhancements that should have either been included with ESRI's 9.2 releases (or 8.3, 9.0, or 9.1 in some cases) or addressed in service packs - that is, now that ESRI is actually releasing new features, not just bug fixes, in its patches.

Like I said, most of the highlights were covered in Jack's response to the UC Questions and Answers that was released last week, but here are a couple of developments that really caught my eye (note that I cover most of these, in-depth, in the commentary below and also that there are many more developments that didn't make it into the plenary that I cover in-depth below):

• More Transparency Within ESRI's Support System
• Release of ArcGIS Server JavaScript and REST APIs
• Enhancements to ArcGIS Server's Tile Generation Capabilities
• New Permissions Management Capabilities Added to ArcGIS Server
• Better Print Support for ArcGIS Server
• New Mobile Client for ArcGIS Server
• Support for PostgreSQL in ArcSDE 9.3

Tuesday, June 19th

ArcPad 7.1 - An Introduction

I've been working with the 7.1 beta for a couple of months now, and, because of this, was excited to hear a bit more from the ArcPad team about the upcoming release. First of all, I'll say that ArcPad is improving dramatically. The project was pretty stale for awhile, but seems to be picking up development momentum (which it really needs). The good news for ESRI is that there isn't (and hasn't been) much real competition out there in the GPS software arena, so they haven't lost much ground. Actually, it seems like they are - amazingly enough - still in front of the small group of GPS software vendors. Sure, many users love Terrasync and its easy-to-create forms, but most major organizations that use GIS are ESRI shops and need: 1) something that will fit into their overall GIS infrastructure (read: enterprise) strategy and 2) software that is released and supported by a company that they know will not go under in a year's time.

ArcPad 7.1 introduces a new extension for ArcGIS Desktop. They're calling it "ArcPad Data Manager" and, unlike ArcPad 6.x's desktop extension, it is installed (and, thankfully, un-installed) with ArcPad (so it is not installed when ArcGIS Desktop is installed). This new tool adds quite a bit of functionality to the check-out/check-in process, which is a process that, although it always seems to work well for me in 7.01, is always the most difficult part of working with ArcPad for users who don't have a lot of technical experience. I can't tell you how much time I've spent walking users through the check-out/check-in process (literally over and over), even after developing comprehensive step-by-step documentation that shows them exactly how to do it. I'm thinking, though, that this new data manager extension will solve many of these problems, as the status of checked out data are no longer stored locally on a computer. They are, rather, stored within the checked-out data themselves, which means that a user can check data out from one computer and then check it back in using another computer and it doesn't have to live in the same exact directory location to be successfully checked back in. On top of this, check-in can now be initiated from within ArcMap, allowing a user to browse to the data that need to be checked-in rather than having to make sure that the data that need to be checked-in live in the exact same place in the directory as the data that were checked-out. Although this may not sound earth-shattering, it is a very welcome enhancement that will likely save me hours of troubleshooting by telephone.

To describe the new ArcMap extension more fully: When the "ArcPad Data Manager" extension checks data out of a geodatabase, the data are now placed in a single xml file, with an .axf extension. This allows for all of the data to remain in a single file. The .axf format uses SQL Server Compact Edition to store its data, which allows for much more to be done with the data in ArcPad.

The biggest enhancement that this new format allows is the support of related data within ArcPad. We were shown a demonstration of creating a single geometry feature in ArcPad and then collecting attributes that lived in related tables. This is obviously a huge development for ArcPad, and is an issue that the ArcMobile team doesn't seem to have a clear solution for (more on this in the ArcGIS Mobile - The Road Ahead and Developing Applications with ArcGIS Server Mobile SDK sections below). The important thing to get across here is that these relationships are supported in the data (the .axf file), not in the forms. This should allow for easier transition in and out of a geodatabase without having to custom-design applications. Another important development is that the check-out and check-in tools will now be available in ArcToolbox, allowing for easy scripting of the check-out/check-in process with Python. I'm personally hoping that I'll be able to make the whole check-out/check-in process completely invisible to our organization's mobile users, as - like I mentioned before - this is the biggest problem that they run into when working with ArcPad today.

A couple of miscellaneous other enhancements that were discussed in the session, in no particular order:

• On check-out, you can now name the .apm that is created whatever you want.
• Data that are checked out can now be optionally encrypted and password-protected.
• Forms will no longer be overwritten on check-out, unless specified by the user. This means that customized forms can be built to support data collection and used without the user having to manually move and overwrite the form that is generated on check-out out of the geodatabase.
• If the geodatabase enforces unique identifiers, this will be supported in ArcPad, as well. As a sidenote, it should be easy to generate GUIDs with the .NET Compact Framework and SQL Server Compact Edition, so if you use GUIDs a lot, it should be simple to implement in the new ArcPad.
• New editing features introduced.
• Better GPS height support.
• The ability to capture features using a traverse method.
• Better support for cameras.
• Better search capabilities within ArcPad:
  • The ability to pre-configure query forms within ArcPad Application Builder Studio is now included.
  • A more intuitive search interface, including support for basic SQL.
• Support for read-only features.
• Enhancements to label tools.
• The ability to set a snapping tolerance, per layer, within ArcPad.

Another question that kept on coming up, over and over, at the UC centered on the seeming overlap between the ArcGIS Mobile and ArcPad products. Here's how it was described to me: ArcMobile is meant to serve as an easy-to-use client for ArcGIS Server. It can connect to servers in real-time and has some GPS functionality integrated. It does not, however, support many of the advanced features that ArcPad does, including support for connected devices (rangefinders, etc.) and will never have as much GPS support and as many advanced features as ArcPad. The ArcMobile team is really looking to appeal to a specific market, those who use ArcGIS Server and have a need to collect some basic data while in the field. To me, ArcPad really shines when there is a need to collect complex data into a geodatabase without having to custom design an application. ArcMobile seems to be better suited in cases where a very-targeted application that communicates with ArcGIS Server is needed and more flexibility is desired for developing on the client-side.

For me, I'm interested in both technologies. I love to have choices when hashing out the requirements of an application and deciding on a solution, and I truly think that my organization will be using both ArcMobile and ArcPad, depending on the needs of each specific project.

ArcGIS Server - The Road Ahead

There were tons of upcoming enhancements to ArcGIS Server 9.3 announced at the UC. Most of these, however, for me fall into the "should've been there at 9.2" category. I don't want to always be negative, especially when I think that ArcGIS Server is a great product that has improved dramatically over the last two service packs (even with some of the workarounds that are still required, yet-to-be-fixed bugs, and lack of cohesive and comprehensive documentation). ESRI needs to realize, though, that they can't continue to release beta products as runtime.

One glaring example of this is the documentation for ArcGIS Server 9.2; it basically didn't exist when 9.2 was first released and still lacks some critical pieces. What's the solution (other than trying to catch up on the documentation as soon as possible)? I've got a couple of ideas:

1. Loosen the release cycle, opening it up so that the different product teams can release when their product is actually ready. They've done this with ArcGIS Explorer (which is another product that should've held off on its initial release), and, because of this, the team is on a very aggressive release schedule and is able to quickly respond to bugs and feature requests.
2. Open up the beta program to a larger audience. There are obviously problems with ESRI's beta process when their initial releases don't perform as advertised. Opening up the beta to more real users could dramatically increase the amount of feedback coming back to ESRI, allowing them to truly test their product before initial release.

Anyway, I'm not going to solve all of the world's problems today, so enough about that and back to the topic at hand: upcoming developments in the ArcGIS Server product.

First of all, ESRI announced in the UC Q&A that they realized that ArcGIS Server 9.2 shipped without proper documentation (see my rant in the first paragraph of this section above), especially for developers. In this session - and at a few other times during the UC - they mentioned this again and made it clear that they are working hard to improve their online help and documentation. This is, of course, very welcome.

Another very welcome development is ESRI's decision to release JavaScript and REST APIs for ArcGIS Server at 9.3. This is another development that was first announced the week before the UC, and then again at the plenary, and then mentioned/discussed several times during the course of the conference. But I never got sick of hearing about it. Here are some more details:

• The REST API will be completely securable, just like any other service that is served by ArcGIS Server.
• The REST API supports querying, geocoding, and geoprocessing.
• The JavaScript API(s) will allow for easy development and deployment of lightweight web applications that consume both ArcGIS Server services and third party data services, like data consumed from ESRI ArcWeb Services, Google Maps, Mapquest, Microsoft Virtual Earth, and Yahoo! Maps.
• A token-based authentication system for JavaScript clients is also being released.

Further enhancements that are coming to ArcGIS Server 9.3 follow, in no particular order:

• Enhanced AJAX Support
  • Microsoft's ASP.NET 2.0 AJAX 1.0 extensions will be fully supported in the 9.3 release of ArcGIS Server. This includes support for Update Panels. With this addition, developers now have a choice as to which callback model they use. It will still be possible to use the current ASP.NET 2.0 callback implementation that is used in 9.2, but I am recommending that all of our applications migrate fully to ASP.NET AJAX, as it is a powerful, clean, and easy-to-use implementation. As an aside, as many probably already know, it is currently possible to use ASP.NET AJAX, but there are a lot of workarounds to get some of the ArcGIS Server controls (especially the map control) to work correctly when wrapped in an Update Panel.
  • Partial page rendering model supported (comes with ASP.NET 2.0 AJAX 1.0).
• New security management module will be included in ArcGIS Server Manager. This will allow administrators to set up access permissions for both services and applications using groups and users. I don't think that roles are supported, although I'm not positive about this. Of course, this is all currently doable in 9.2, but you have to manually configure NTFS permissions - if using Windows Integrated or Active Directory forms authentication - and manually make changes to the web.config. See the "Miscellaneous Questions/Answers and Other Tips that I Picked Up" section below for a little tidbit of helpful info that I picked up when speaking with the ArcGIS Server development team about configuring a web application and associated services to use Windows Integrated Authentication.
  • Ability to manage users per application and/or per service.
  • Can ensure that all communication with ArcGIS Server is encrypted (again, already doable in 9.2 but supposedly easier in 9.3).
  • As mentioned above, new token based authentication system for JavaScript clients.
  • Can work with Windows authentication or use forms authentication and store user information in SQL Server database.
• A big development in the open standards arena is that ESRI, with ArcGIS Server 9.3, will introduce support for GML, WCS (1.0 and 1.1), and WFS and improve its implementation of WMS. Unfortunately, though, for ArcGIS Desktop users the Data Interoperability Extension is required to access WFS services. To access WFS in ArcGIS Desktop, you must *install the Data Interoperability Extension (it doesn't have to be licensed/activated).
• At 9.3, Image Server will become an extension of ArcGIS Server. From what I was able to gather, this does not, however, move Image Server from Category B to Category A, so licensing for enterprise partners is not necessarily going to change with this release. That said, from talking with several ESRI employees, it does seem like they are considering eventually moving Image Server to Category A. An image service:
  • Can deliver data or pictures.
  • Is, from what they are saying, very efficient at serving WCS and WMS.
• Major enhancements to ArcGIS Server tile caches are coming at 9.3. As you can tell from earlier posts and comments that I've made on this subject, this is an absolute must for my organization. We have had major problems developing tile cache for many of our projects that cover large areas and need cache at large scales (e.g. half of the world at many scales, the largest being 1:1250). Actually, we haven't been successful in generating this particular cache, even using several servers for weeks at a time. Well, ESRI says that they've fixed many of the tile cache generation problems at 9.3. Here are some of the enhancements that will be coming out:
  • Ability to generate tiles that match the tile format for ArcWeb Services, Google Maps, and Microsoft Virtual Earth.
  • On-demand tile generation: This enhancement, if it works as advertised, will likely fix many of our problems, although it's not the perfect solution for our organization. This basically allows you to generate tile cache up front only for areas that you foresee users actually zooming into. For areas that may not get used as much (or at all), you don't have to generate tiles, but if a user zooms into them tiles are generated on-the-fly and then added to the existing cache for future users. This is a decent solution, although it's not always easy for us to predict where a user's interests lie, and I'm afraid that generation of tiles on the fly is going to be horrendously slow.
  • Generate tile cache based on extent of a feature: This is another *very welcome enhancement, as it should, theoretically, allow us to more effectively partition out and track our tile generation. One of the major problems that we currently have is that when cache generation fails for a large cache build operation, it is often difficult to discover exactly where it failed and then reinitiate the build from that point.
• Some user interface enhancements (for applications built using the wizard in ArcGIS Server Manager) coming at 9.3 include:
  • Zoom previous, zoom next buttons with another tool for toggling an overview map.
  • Map tips will be able to contain static text, images, links, and JavaScript.
  • A new print task will be available in ArcGIS Server Manager and will include the ability to work with map elements.
• Enhancements to the Web ADF will allow developers to take advantage of session state.

Some very exciting news above! I look forward to getting my hands on the new release.

Deploying and Using ArcGIS Explorer

This technical workshop with Bernie Szukalski presenting ArcGIS Explorer was one of the best of the conference for me, if only because it's always nice to hear from someone who is excited about the topic that they're presenting. I've played around quite a bit with ArcGIS Explorer and have been encouraging our organization's GIS users to use and deploy it since the last build (380) was released, but it was still great to see some of the tweaks and operations (tasks) that the interface supports. It was also good to hear a bit about the development environment surrounding ArcGIS Explorer and get some of my questions answered by those who work on the project.

The ArcGIS Explorer development team is not constricted to the standard ESRI release cycle, so they've already had four releases since the initial release with the overall 9.2 release and they plan on releasing a new build (with feature enhancements, feature additions, and bug fixes) every two months. The next release is expected within the next couple of weeks.

As this was more a "show me your product" type of session, I didn't get much in the way of technical news from it. I did, however, get more in the Extending ArcGIS Explorer Using the ArcGIS Explorer SDK session, which is covered below.

Wednesday, June 20th

How to Develop Rich, Web 2.0 Style Flash Mapping Applications Using ArcWeb Services

I wasn't expecting to get too much out of this technical workshop, as, I'll be honest, I haven't paid much attention to ArcWeb Services as it has really come of age over the last year or so. Well, I'll tell you sheepishly now that I was very pleasantly surprised. ArcWeb Services seems to be another valuable tool in the expanding web map industry, and I'll be experimenting with it more shortly.

One of the main reasons that I want to spend some time working with ArcWeb Services, other than the product itself, is that it was mentioned in this session that ArcWeb Services and ArcGIS Server will likely be merged at some point. There are many similarities between the two, including, with the upcoming 9.3 release of ArcGIS Server, the support of JavaScript and REST, and the merge seems pretty logical to me. It seems that most of what ESRI is releasing (ArcGIS Explorer, ArcGIS Online, ArcMobile, ArcGIS Web Services, etc.) and spending most of their money on are technologies that either currently fit into ArcGIS Server or will likely fit into ArcGIS Server in the near future. Is this bad news for ArcIMS? Well, ESRI says that they're going to continue to support it, but why would you want to stick with a product that wasn't being actively developed?

ArcWeb Services 2.0 is currently in beta 1 and can be accessed via the Flex or JavaScript APIs. Beta 2 is expected to be released at the end of July, and will include some major feature additions, including support for consumption of ArcGIS Server services, ArcIMS services, and GeoRSS. In the near future, ArcWeb Services will also support WFS, WMS, and more formats and the use of geoprocessing tasks.

Some benefits that I can definitely see to using ArcWeb Services and its APIs:

• Support for 13 projections, all available client-side.
• Access to loads of data for thematic mapping.
• The ability to upload your own data to ESRI servers and then consume them in an ArcWeb Services application.
• Support for vector printing - this is a biggie that ArcGIS Server doesn't seem to get.

To test drive ArcWeb Services yourself, go to the ArcWeb Services site and sign in with your ESRI global login. You will then be able to apply for a free 90 day trial which will give you access to the APIs and underlying data and services. After the 90-day trial, cost is calculated based on a per transaction model (somewhere around $1250 for 100,000 credits).

Here are some other resources that may be helpful in your evaluation:

• ArcWeb Explorer 2.0 Lab
• 2007 Developer Summit Presentations

Designing, Deploying, and Using Cached Map Services

I sneaked in on the last part of this technical workshop, as when I showed up initially there was a sign on the door that told me the session was full. Looks like others are having difficulty building cache too, huh (note that you can see some of my specific gripes about building tile cache in the ArcGIS Server - The Road Ahead section above)? We've had to learn most of the lessons the hard way, so I was hoping that this session would help us avoid further costly mistakes.

From what I was able to catch, it seemed like a pretty productive session, with lots of tips and insights coming from the ArcGIS Server team. I couldn't help but think, though, how much time, energy, and money ESRI could save their customers by getting these tips out in the form of online help or blog posts. Seriously, though, if ESRI is committed to helping their customers build and maintain an enterprise GIS, shouldn't they make their knowledge on how to make their software work correctly publicly and readily available?

Here are the tips that I caught in the short time that I was in the session:

• ArcGIS Online uses multiple tile servers, all of which have local .mxds that are accessing local data (stored in file geodatabases).
• Once you build a cache, you can get rid of your original data, as it's no longer needed or used by a service.
• If using the "generate cache" command, zoom into the extent that you want to generate cache for, set the max scale, and then start the cache build.
• There is a new blog post with a Python script to help in building cache coming out on the ArcGIS Server team blog this coming week.
• Enhancements to tile cache generation at 9.3 will not "break" existing caches, they will, however, allow you to build cache more efficiently.

Extending ArcGIS Explorer Using the ArcGIS Explorer SDK

This session had some pretty good demonstrations on creating custom tasks for ArcGIS Explorer included, although it was really more valuable as just a deeper introduction into what ArcGIS Explorer is meant to do and how it works. The session started out introducing the basics of an .nmf file, which is an xml file that ArcGIS Explorer uses to store everything. Being xml, .nmf files can easily be programmatically created and edited, and can contain everything from layers (including symbology, etc.) to ArcGIS Explorer tasks. They can basically contain as little or as much information as you want, and, since they are just xml, editing .nmf files is an easy way for users to create "custom" ArcGIS Explorer applications without any programming at all.

If you want to further extend an ArcGIS Explorer application, though, you are going to have to develop custom tasks. Tasks are designed to be very lightweight. They serve as a client-side interface to gather input from users, present this information back to them, and interact with other tasks, if needed. The bulk of a task's processing work gets done on the server.

To help developers design custom tasks, ESRI has released an API that allows developers to interact with ArcGIS Explorer. Note that the SDK is .NET only.

Here are some tips for those of you who are developing tasks for ArcGIS Explorer:

• As drawing lots of results takes time, set visible to "False" to begin. This dramatically improves the overall user experience.
• Currently, the polygon property does not have area. This is because of difficulties with WGS84 and 3D.
• ArcGIS Explorer returns hidden layers, so make sure that you check for these when iterating through layers.
• You can override "OnIdle" to bring real time updates of data. There are currently no events exposed in the API to do this, but it is possible.
• Result::Description does honor HTML tags, which means that you can use CSS, JavaScript, etc. in the description.

One more development that is coming with the next build of ArcGIS Explorer is exposure of OpenGL, which should allow for lots of exciting things to come.

Thursday, June 21st

ArcGIS Mobile - The Road Ahead

I wasn't able to make it to the 2007 ESRI Developer Summit, so I missed quite a bit of news on the ArcGIS Mobile front. Because of this, I was especially excited to hear more information about ESRI's plans for the SDK. Most of the session's time, however, was spent on the new ArcGIS Mobile client (it looks like they're calling it "ArcGIS Mobile Editor", although don't quote me on that). I was a little disappointed by this, but it was interesting to see some of the demo apps that they put together for the new client, as most of the functionality that was demonstrated is available now with the existing 9.2 SDK. After the session, I was told that there should not be any major changes to the SDK (other than additions) from 9.2 to 9.3, so there shouldn't be much work involved when migrating to 9.3.

As for the SDK, there are some enhancements coming at 9.3:

• Support for more base layers, including large, non-edit vector datasets and tiled ArcGIS Server map services.
• More street routing and navigation functionality exposed.
• An updated geometry and sketch model, including making it easier to create features with GPS (this can be done with the 9.2 SDK, but it is difficult).
• Better standalone and related tables integration. It was not clear how they are going to deal with related data, although I did ask.
• Support for Windows Mobile 6.0 and Linux Mobile Data Web Service.

Like I mentioned, the majority of the session was spent on the upcoming ArcGIS Mobile Editor. Here are some miscellaneous features that were discussed and demonstrated:

• Target release date: January 2008, along with the rest of the 9.3 release.
• The client is targeting Windows Mobile devices.
• Disconnected, occasionally connected, and always connected use cases will be supported.
• Will not support related tables in initial release.
• The team has really focused on usability and the user interface shows this. It is very simplistic, without any icons at all (it uses text in place of icons). It was interesting that they mentioned looking to the iPod for inspiration when building the user interface.
• Allows users to both collect and update features.
• Has full GPS editing capabilities.
• Can synchronize directly with ArcGIS Server with virtually any kind of connection - including a satellite phone connection, WiFi, CDMA/GSM, and wired connection - which allows for real-time tracking of either a GPS device (via the tracklog) or field work completed.
• Feature search functionality and ability to connect to web service to search, as well.
• Release of a new module for ArcGIS Server Manager which allows for development and deployment of ArcGIS Mobile services and applications.
• When a mobile service is accessed by ArcGIS Mobile Editor, geodatabase subtypes and domains are automatically brought with it onto the device.
• Uses local caching for "non-operational" data (data that are not being edited). These should not be sent over wireless, but, rather, loaded onto the device.
• Compressed vector layers are supported for static viewing.
• ArcGIS Online will soon have services specifically designed for consumption by ArcMobile applications, including: StreetMap, satellite imagery, and topopgraphic data.
• Wireless synchronization can be tweaked to fit specific cases (e.g. synchronize when a connection is available, synchronize when a WiFi hotspot is available, or automatic synchronization in real-time).

Managing Distributed Data with Geodatabase Replication

This session, although it covered quite a bit of material that was review for me, did help quite a bit. At my organization, we had a couple of problems early on with replication that have now been solved. Overall, from my experience geodatabase replication seems to work fairly well, and it sounds like there are quite a few improvements coming in 9.3 that will make it even more solid.

Again, I'm just going to list a few of the highlights from this session, as a lot of the session was just reviewing the fundamentals of using geodatabase replication:

• No version is needed on a simple child replica. Having a version will only decrease performance of the geodatabase and the replica.
• With the current implementation of geodatabase replication, metadata are copied over when the first replica operation is run. After that, metadata changes are not reflected in the replica. This may change in future releases, although the enhancement is not currently slated for the 9.3 release.
• There are tools that allow you to apply most kinds of schema changes across replicas. Applying schema changes is a two step process:
  1. Compare replica geodatabases directly in a connected environment.
  2. Export XML and then import to apply changes to geodatabase that needs the schema changes.
• Geodatabase replication can be integrated into larger application through APIs. There are two APIs, one for coarse-grained control (GeoDataServer object model), the other for more fine-grained control (Pre-ArcGIS 9.2 object model that works only on geodatabases that live on the Local Area Network).
• Enhancements coming at 9.3:
  • Replication will be extended to allow one way replication to file and personal geodatabases.
  • Better support for synchronization of large (greater than 1 GB) delta files.
  • Improved logging for easier troubleshooting of issues.

Lastly, at the end of the session it was announced that a whitepaper, titled "Managing Distributed Data" will be available on the ESRI Support site soon.

Developing Applications with ArcGIS Server Mobile SDK

This was another session that I didn't get a lot out of. Maybe I was just getting tired? Here are a few random notes:

Overview of building mobile applications for accessing ArcGIS Server services:

1. Build your geodatabase.
2. Author a mobile map.
3. Publish your map as a mobile service.
4. Design mobile application.
5. Deploy mobile solution.
6. Synchronize mobile solution.

Tips for developing mobile applications for accessing ArcGIS Server services:

• Use MapCache Generator tool on desktop to generate mobile map cache.
• Use Request_Completed event to monitor synchronization requests.
• Use the State and Notifications Broker API and be smart about how connected you are (Microsoft.WindowsMobile.Status).

Miscellaneous Questions/Answers and Other Tips that I Picked Up

So, as all of you who have been to the UC know, a lot of what you learn comes outside of the official workshops and technical sessions. The vendor area holds a plethora of resources in the form of technical expertise - especially for developers - that are available to help troubleshoot specific issues and field questions, and you can always learn something when talking to colleagues. Here are some miscellaneous anecdotes that I discovered during my time here in San Diego. Where appropriate, I've given credit to the originator of the information:

ArcGIS Explorer

• Can an .nmf be placed on a file server and not be downloaded/cached locally when a user opens it from his/her workstation?
  • It depends on the browser behavior. ESRI hasn't really looked into this, but they are going to and will come out with some guidance/recommendations on how to make various browsers handle .nmf files in this way. If your organization uses Internet Explorer, this can likely be set by group policy.
• Have any formal tests been done to help gauge the bandwidth cost of using ArcGIS Explorer on a corporate network?
  • No formal tests have been done, but because of the way that ArcGIS Explorer separates data into layers (which is a good thing, right?), it probably consumes more bandwidth than Google Earth. ArcGIS Explorer is designed to be a broadband application and will, therefore, not run with less than 100 kb/s bandwidth. For a good experience, there should be at least 200 kb/s of available bandwidth per user.

ArcGIS Server Performance

• Generating Tile Cache
  • Just a tidbit of information: to generate tile cache for the globes that are served through ArcGIS Online, ESRI used three quad CPU servers to build each of the caches. It took them two weeks to build each cache.
  • When generating tile cache for use in ArcGIS Explorer, it is possible (and recommended) to generate 2D cache and then associate it with a globe. Draping will happen on the fly, and this allows for cache to be generated much faster and with much less overall processing.
• Using Multiple DNS Entries to Serve ArcGIS Server Tile Cache
  • Lately, I've been working hard on improving performance of some of our web applications. One tried and true method that web developers live by is registering multiple DNS entries to serve out resources (CSS, Images, JavaScript, etc.), as this helps in getting around HTTP 1.1's two connections per-browser limit. I haven't been able to make this work for our tile cache yet, so I posed the question to one of ArcGIS Server's developers. His idea was to specify the names of the servers in the code and then use JavaScript to cycle through them on each image request. We then went over to discuss it with the product lead who told me that a blog post will be out in the next couple of weeks that will detail how to set this up.

ArcGIS Server Security

• At 9.2, it isn't possible to use Windows Integrated pass-through authentication from an ASP.NET ArcGIS Server application to a database server to, for example, limit edit access to a service based on an Active Directory logon. There are ways, however, to work around this and basically get the same thing. Note that what follows isn't pretty:
  • You have to place the service in a directory that has NTFS permissions set on it and make some modifications to the web.config. If a user that is not allowed access to the service goes to the web application, they will not see the service.
  • In this way, you can 1) create two different services that access the same SDE data (one for editing, one for viewing) and 2) restrict access to the services using NTFS permissions and modifications to the web.config. After doing this, the user will see only the service that they have access to in the application.
  • However, in addition to this you have to programmatically hide the "Edit" task that displays in the web application if a user does not have edit access to any services (and, if you want to make it even more complicated, you have to create and programmatically hide or display custom edit tasks per Active Directory group if you have multiple services that are editable by different groups of users).

Web Map Optimization (thanks Roland!)

• Cartographic representations of data draw faster in web maps than explicitly-defined symbology representations.
• It is good practice to convert .bmp to .emf before publishing maps. There is a style set, titled "ESRI Optimized Style File" that is available for use. This set contains styles that have been designed specifically for display on the web.
• Where possible, it is good practice to simplify your geometries before serving them through a web map application. For example, the "Simplify Polygon" command honors topologies while generalizing the geometry of a polygon feature class.
• When building a map that is going to be served through ArcGIS Server, define your scales initially and then group data that display at specific scales (scale-specific rendering) into group layers. ESRI has defined its standard scales, and loads their "StandardScales.txt" into each of their .mxd projects before doing anything else.

From ScottGu's blog: "ASP.NET AJAX is available for free, and can be used with ASP.NET 2.0 and VS 2005.  It is a fully supported Microsoft product, and is backed by a standard 10 year Microsoft support license (with Microsoft Product Support available via phone 24 hours a day x 7 days a week)."

Also exciting is that the new release of the AJAX Control Toolkit - a set of free, shared-source controls for ASP.NET - has four new controls, including the Calendar Extender, Masked Edit Extender, Tab Container, and Auto Complete Extender. From the looks of it, development on the Control Toolkit will continue even after this final 1.0 release. You can download the Control Toolkit from CodePlex.

So, this seems to be the process for near-future development on the AJAX project (even though they have apparently already started work on the ASP.NET AJAX 2.0):

• Bleeding-edge controls are released through the "Future" CTP, available for download at the
ASP.NET AJAX site
• After these controls are tested, they make it into the major release of the Control Toolkit

Another important tip that Scott mentions in the release announcement: If you've worked with pre-release versions of ASP.NET AJAX in Visual Studio, it's a good idea to delete the cached Visual Studio schema files for control markup. This will ensure that Intellisense is properly working in the Visual Studio environment. To do this:

• "With Windows XP you do this by deleting all files in this directory: c:\Document and Settings\[User]\Application Data\Microsoft\VisualStudio\8.0\ReflectedSchemas
• With Windows Vista you do this by deleting all files in this directory: c:\Users\[User]\AppData\Roaming\Microsoft\VisualStudio\8.0\ReflectedSchemas"

The Microsoft ASP.NET AJAX development team released their first (and they say, only) release candidate yesterday on December 15th. Here's the announcement.

They have also updated the AJAX Control Toolkit - a collection of ASP.NET AJAX-enabled controls - and made it available for download on CodePlex.

I've written a couple of times about my experiences with the ASP.NET AJAX extensions, and know that I will continue to use them in my web apps. Overall, I have been impressed with the speed of development on this project and the number of feature enhancements and issues addressed over the last several months. I do, however, look forward to that final release, as it'll be nice to stop worrying about upgrading my applications for each new release (I know, I know, my fault for using pre-production software).

On a slightly different, but related, note: I briefly mentioned Microsoft's release of Visual Studio 2005 SP1 last week. I didn't, however, have time to really dive into the changes that SP1 brought to the IDE (Visual Studio, all editions, and Visual Web Developer). Over the last week, though, I've noticed a major improvement that warrants mention, especially to ASP.NET AJAX developers; much improved Intellisense support for the ASP.NET AJAX extensions!

Listening to Lucinda Williams - Live @ The Fillmore...
My Nose is In Donna Tartt - The Little Friend...

A quick one: Up until this latest version of the ASP.NET Ajax extensions, one of the glaring "bugs" (if you want to call it that) with the CascadingDropDown control was that it returned the backend value of the dropdown rather than the actual selected display value. This caused problems when trying to access the SelectedValue. The only way to quasi-cleanly work around this was to manually edit the associated javascript.

Well, I just found out that this problem was addressed in the latest release. You can now access the SelectedValue of a CascadingDropDown control without having to implement a messy workaround:

ddlCars.SelectedItem.Text

Not anything ground-breaking, but a very welcome fix.

Listening to Broken Social Scene - You Forgot It in People...
My Nose is In Daniel Quinn - Ishamel...

I've been using Microsoft's ASP.NET AJAX implementation since the early "Atlas" beta days, but am just now starting to feel comfortable enough with its stability to use it in actual projects. Most of the sample controls that are included with the framework are remarkably easy (and fun) to setup and use. It's pretty amazing how much usability can be added to an application through the use of AJAX.

Well, I recently set out to use the CascadingDropDown control on one of my sites, but found that I ran into several different issues. The example that is included with the ASP.NET AJAX Control Toolkit grabs its values from an XML dataset. I, however, wanted the controls to grab their values from tables in a SQL Server 2005 database. Luckily, there is a pretty decent walkthrough for setting up the control with a database available on the ASP.NET AJAX webpage. I found it incomplete in a couple of important areas, though, so I thought that I would put up my version of a walkthrough for those who find themselves running into issues with implementation.

Walkthrough

I decided to use ASP.NET 2.0's datasets feature, with the new TableAdapters functionality to provide access to the data in my tables. Here are the steps to setting up the data access side:

1. Create a new dataset (right-click in Solution Explorer, "Add New Item...", Dataset).

2. Configure your TableAdapter(s) using the "TableAdapter Configuration Wizard".

• If you don't have an existing connection to your database, you will need to create one in the first dialog of the wizard.
• Build as many TableAdapter queries as you need. You should have a query built for each CascadingDropDown control that you'll be using in your application.
  • The first query will likely be a simple select query on a single column (example, as presented in the ControlToolkit and in the code below: make of a car).
  • The next query(ies) will need to "GetData" by a certain parameter that was returned from the previous query(ies) (example: model of a car, filtered by the make that was returned from the first query).
  • You will probably want to sort by one or more fields in your query(ies), as well.
  • Make sure that you validate your query(ies) to make sure that they return the proper records before moving on.
• If you are grabbing data from multiple tables, you will have to add each table into the dataset and build TableAdapter(s) for each one.

Now that you can access your data and return the necessary values through the dataset, you are ready to build your Page/Web Methods. I chose to make calls to Web Methods, as a known limitation of using Page Methods in the ASP.NET AJAX beta 1 release (note: they just released beta 2, but I'm not sure if this issue has been addressed in the new release) is that if you use Page Methods, the code has to live in the .aspx page.

1. To create a new Web Method, right-click on the directory where you want your code to live, select "Add New Item...", select "Web Service" from the templates, check the "Place code in separate file" option, and click "Add".

2. Make sure that you have all of the following namespaces, along with the [Microsoft.Web.Script.Services.ScriptService()] entry at the top of your .cs, assuming, of course, that you are programming in C#:

using System;
using System.Web;
using System.Collections;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Web.Services;
using System.Web.Services.Protocols;
using AjaxControlToolkit;
using System.Data;
using System.Data.SqlClient;

/// <summary>
/// Summary description for DataService
/// </summary>
[WebService(Namespace = "http://yourdomain/webservices")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[Microsoft.Web.Script.Services.ScriptService()]
public class DataService : System.Web.Services.WebService
{

    public DataService ()
    {
        //Uncomment the following line if using designed components
        //InitializeComponent();
    }

3. build the Web Method(s). Following are two sample web methods:

[WebMethod]

    public CascadingDropDownNameValue[] GetMakes(string knownCategoryValues, string category)

    {

        CarsTableAdapters.MakeTableAdapter makeAdapter = new CarsTableAdapters.MakeTableAdapter();

        Cars.MakeDataTable makes = makeAdapter.GetMakes();

        List<CascadingDropDownNameValue>values = new List<CascadingDropDownNameValue>();

        foreach (DataRow dr in makes.Rows)

        {

            string make = (string)dr["Make"];

            int makeId = (int)dr["MakeID"];

            values.Add(new CascadingDropDownNameValue(make, makeId.ToString()));

        }

        return values.ToArray();

    }

[WebMethod]

    public CascadingDropDownNameValue[] GetModelsByMake(string knownCategoryValues, string category)

    {

        StringDictionary kv = CascadingDropDown.ParseKnownCategoryValuesString(knownCategoryValues);

        int makeId;

        if (!kv.ContainsKey("Make") || !Int32.TryParse(kv["Make"], out makeId))

        {

            return null;

        }

        CarsTableAdapters.ModelTableAdapter modelAdapter = new CarsTableAdapters.ModelTableAdapter();

        Cars.ModelDataTable modelTable = modelAdapter.GetModelsByMake(makeId);

        List<CascadingDropDownNameValue> values = new List<CascadingDropDownNameValue>();

        foreach (DataRow dr in modelTable.Rows)

        {

            values.Add(new CascadingDropDownNameValue((string)dr["Model"], dr["ModelID"].ToString()));

        }

        return values.ToArray();

    }

The first method simply grabs the "Make" and "makeId" (key field) from the Make table and returns them to an array, displaying the Makes in the first dropdown. The second method parses out the valid car models, using the "makeId" to identify the valid returns, and displays them in the second dropdown.

The next step includes adding the dropdown list(s) to your ASP.NET page and pointing it/them to your Web Method(s). Here is the code to create two dropdown lists and hook them up to the Web Methods. Make sure that you turn EnableEventValidation off in the page directive (see code below). If you don't, your CascadingDropDowns will not populate and you'll get a helpful [Method 500 error]:

<%@ Page Language="C#" AutoEventWireup="true" EnableEventValidation="false" %>

<%@ Register Assembly="AjaxControlToolkit" Namespace="AjaxControlToolkit" TagPrefix="AjaxToolkit" %>
<html xmlns="http://www.w3.org/1999/xhtml">
<head id="head1" runat="server">
    <title>CascadingDropDown Example</title>
</head>
<body>
    <form id="form1" runat="server">
        <asp:ScriptManager ID="ScriptManager1" runat="server">
        </asp:ScriptManager>
        Make:
        <asp:DropDownList ID="ddlMakes" runat="server">
        </asp:DropDownList><br />
        <br />
        Model:
        <asp:DropDownList ID="ddlModels" runat="server">
        </asp:DropDownList>
        <AjaxToolkit:CascadingDropDown ID="CascadingDropDown1" runat="server" TargetControlID="ddlMakes"
            Category="Make" PromptText="Select a Make" ServicePath="CarsService.asmx" ServiceMethod="GetMakes"
            LoadingText="Loading Makes..." SelectedValue="" />
        <AjaxToolkit:CascadingDropDown ID="CascadingDropDown2" runat="server" TargetControlID="ddlModels"
            Category="Model" PromptText="Select a Model" ServicePath="CarsService.asmx"
            ServiceMethod="GetModelsByMake" LoadingText="Loading Models..." SelectedValue=""
            ParentControlID="ddlMakes" />
    </form>
</body>
</html>

Resources

Some helpful resources that I used:

• Most of the core code came from Shawn Burke in his "Using CascadingDropDown with a Database" walkthrough.
• Gabriel Flores' CascadingDropDown walkthrough.
• "Building and Using a 3-tiered Data Architecture in ASP.NET 2.0" series on http://www.asp.net.

Oh, and by the way, the code snippets in this blog entry were formatted using Omar Shahine's "Insert Code for Windows Live Writer" extension.