AIR multiple file uploads: drag n drop to client zip to server upload and unzip

25 03 2009

The aim of this post is to get an AIR application to take a collection of files dragged n dropped onto it, zip them up on the client, and send them off to the server (ASP.NET). 

There are a few things to note here:

  • Flash officially supports files of up to 100MB, trying to ZIP anything larger and you WILL run into server memory issues; Flash will crash. You can use File.upload() for files > 100MB, but trying to ZIP them will cause the crash.
  • ZIPping on the client takes time, 50MB could take from 15s to a minute. The code supplied performs it sync (not async). 
  • client software FZip, server zip : ISharpZipLib 
  • uploading manually (using URLLoader instead of File.upload means in you cannot track the Upload’s progress – as of AIR 1.5).

 

Step 1. The Drag & Drop in AIR

This is quite trivial. Allowing the user to drag and drop as many files/folders onto your application.

Client (AIR) code:

<?xml version="1.0" encoding="utf-8"?>
<mx:WindowedApplication 

	nativeDragEnter="onDragEnter(event)"
	nativeDragDrop="onDragDrop(event)"

	layout="absolute" 

	xmlns:mx="http://www.adobe.com/2006/mxml" >

	<mx:Script>
		<![CDATA[

			import mx.managers.DragManager;
			import mx.events.DragEvent;

			private function onDragEnter(evt:NativeDragEvent):void
			{
				//ensure clipboard contains files
				if (evt.clipboard.hasFormat(ClipboardFormats.FILE_LIST_FORMAT))
				{
					NativeDragManager.acceptDragDrop(this);
				}
			}

			private var _filesToUpload:Array;

			private function onDragDrop(evt:NativeDragEvent):void
			{
				_filesToUpload = evt.clipboard.getData(ClipboardFormats.FILE_LIST_FORMAT) as Array;
			}

		]]>
	</mx:Script>

</mx:WindowedApplication>

 

Step 2. Zipping on the client

The nicest solution I could find for compressing files in AIR to ZIP is FZip.

I tried another library, but ISharpLib didn’t like it, it was giving me EOF file header issues in the ZIP decompression.

So the solution is to take the ZIP byteArray and to send it via the lower level URLLoader class.

NOTE: this uses the FileStream.open is sync mode. You could use FileStream.openAsAsync though it would obviously require listening and handling the corresponding events.

Client (AIR) code [URLLoader.load() solution]:

var request:URLRequest = new URLRequest("http://example.com/Handlers/UploadHandler.ashx");

request.method = URLRequestMethod.POST;
request.contentType = "application/octet-stream";

var loader:URLLoader = new URLLoader();

loader.dataFormat =  URLLoaderDataFormat.BINARY;

loader.addEventListener(Event.OPEN, onFileUploadStart,false,0,true);

//no point listening for progress, because it doesn't work for URLLoader uploads, only downloads
//loader.addEventListener(ProgressEvent.PROGRESS, onFileUploadProgress,false,0,true);

loader.addEventListener(Event.COMPLETE,onFileUploadComplete,false,0,true);

var fzip:FZip = new FZip();

for each (var file:File in model.filesToUpload)
{
	var fs:FileStream = new FileStream();
	fs.open(file, FileMode.READ);
	var data:ByteArray = new ByteArray();
	fs.readBytes(data);
	fs.close();

	fzip.addFile(file.name, data);
}

var bytes:ByteArray = new ByteArray();
fzip.serialize(bytes);

request.data = bytes;

loader.load(request);

 

What about sending extra parameters?

Client (AIR) code:
var request:URLRequest = new URLRequest("http://example.com/Handlers/UploadHandler.ashx?someVariable=" + someVariable);

What about the upload progress bar? 

If you need the progress bar, then you’re best option is writing this zip file to the client and uploading that. 

This also means that if you have optional parameters, you don’t have to append them to the URL but can send them in the data.

This changes the client code to use File.upload().

Client (AIR) code – [File.upload() solution]:

var params:URLVariables = new URLVariables();

params.someVariable = someVariable;
//add other variables here

var request:URLRequest = new URLRequest(model.UPLOAD_URL);

request.method = URLRequestMethod.POST;
request.data = params;

var fzip:FZip = new FZip();

for each (var file:File in model.filesToUpload)
{
	var fs:FileStream = new FileStream();
	fs.open(file, FileMode.READ);
	var data:ByteArray = new ByteArray();
	fs.readBytes(data);
	fs.close();

	fzip.addFile(file.name, data);
}

//get the zip as a byte array
var bytes:ByteArray = new ByteArray();
fzip.serialize(bytes);

//write the zip to a local file
var fsW:FileStream = new FileStream();
//change "tmp.zip" to a better name :)
var wFile:File = File.applicationStorageDirectory.resolvePath("tmp.zip");

fsW.open(wFile, FileMode.WRITE);
fsW.writeBytes(bytes);
fsW.close();

//listen to upload events
wFile.addEventListener(Event.OPEN, onFileUploadStart,false,0,true);
wFile.addEventListener(ProgressEvent.PROGRESS, onFileUploadProgress,false,0,true);
wFile.addEventListener(Event.COMPLETE,onFileUploadComplete,false,0,true);

//start the upload
wFile.upload(request);

 

Step 3. Unzipping on the server

Now, I use SharpZipLib in .NET because it’s Open Source, and because it’s also bundled with WebORB .NET, the remoting gateway that I often use.

If you use the URLLoader.load() option above (from Step 2) – then you must use Request.InputStream to access the parameters

Server (ASP.NET C# Handler – ashx) code: (SharpZipLib extraction)

using ICSharpCodeInternal.SharpZipLib.Zip; 
   public class UploadHandler : IHttpHandler {

        public void ProcessRequest (HttpContext context)
        {
            String someVariable = context.Request["someVariable"];

            ZipInputStream zipStream = new ZipInputStream(context.Request.InputStream);
            ZipEntry zipEntry;

            while (true)
            {
                zipEntry = zipStream.GetNextEntry();

                if (zipEntry == null)
                {
                    break;
                }

                //choose location to extract files to
                string serverFolder = context.Server.MapPath("~/Uploads/");

                FileStream streamWriter = File.Create(( serverFolder + zipEntry.Name));

                int size = 2048;

                byte[] data = new byte[2048];

                while (true)
                {
                    size = zipStream.Read(data, 0, data.Length);

                    if (size > 0)
                    {
                        streamWriter.Write(data, 0, size);
                    }
                    else
                    {
                        break;
                    }
                }

                streamWriter.Close();

            }

            zipStream.Close();
        }
    }

 

 

If you are using File.upload() on the client 

You’re Handler in ASP.NET now will be looking in the context.Request.Files array for the attached ZIP.

Modified Server (ASP.NET C# Handler – ashx) code:

ZipInputStream zipStream = new ZipInputStream(context.Request.Files[0].InputStream);

 

If you need folder structures

To do ZIP folders, you simply add the folder name before the file when you ZIP:

Modified Client (AIR) code:

fzip.addFile(folderName + "/" + file.name, data);

 

And then in the server code, you need to create directories if required:

Modified Server (ASP.NET C# Handler – ashx) code:

//existing code
...
string serverFolder = context.Server.MapPath("~/Uploads/");
string dir = Path.GetDirectoryName(serverFolder + zipEntry.Name);

if (!Directory.Exists(dir))
{
    Directory.CreateDirectory(dir);
}                

//existing code
FileStream streamWriter = File.Create(( serverFolder + ze.Name));

...





Rich clients (Flex/AIR/Silverlight) & LINQ : data across the pipe

19 03 2009

I’ve blogged before about Flex & LINQ, but I’m always learning new things, and I wanted to share some of my most recent findings.

Note: this article focuses on Flex & LINQ, but it could as easily be applied to Silverlight & LINQ as well. Silverlight 3 is set to merge client & server with .NET 4.0, but I believe the issue of how much and when to send data between client & server will continue to be relevant.

First off, why LINQ?

  1. With LINQ on your server layer, you can avoid having to write your server value objects – LINQ to SQL will do that for you from your DB schema – allows you to quickly updates these objects if your schema ever changes (and let’s face it, it will). NOTE: unfortunately, there is no free & easy way to sync your LINQ to SQL context to your schema in VS2008).
  2. You can put all your SQL as strongly-typed code in your DAL rather than having to call the DB to do it via stored procs, which decouples your DB from your application, allowing you to switch databases if you ever wanted to, and keeps all your server code in one place. 
  3. Cleaner and shorter code – easier for you, easier for other developers to understand (as long as they understand LINQ 🙂 )

So, if you’ve decided to take the LINQ option and your a Flex or AIR developer, then there is good news. 

Products such as WebORB.NET and FluorineFx both support .NET 3.5 and will allow your Flex application to send Remote Objects back and forth between the two. Yes you could use Web Services to do communicate with the server, but I’m assuming you’ve read up on the performance benefits of remoting and AMF vs Web Services, and you’re opting for speed.

The question is however, how do you do it?

The LINQ to SQL tool (I’m not going to mention the ADO.NET Entities tool, which is more feature rich, but a touch more complicated) creates you a data context as you know. This remembers the actions you perform on your objects and will propagate the changes back to the DB when you “SubmitChanges()”. However, when you’re working on the Flex/.NET model, you’re now using LINQ on an middle-tier (or even an n-tier) application, and that changes things.

What changes is that the “context” will be a new instance between each request (unless you use session access to the RemoteObjects, which I won’t get into here). LINQ won’t know that you changed something outside this context, and so to Update an object in the database you will need to “Attach()” it back to the current context to save it, along with it’s original but from a new  context. (See examples below).

Now, there are plenty of articles out there on LINQ in the middle tier and LINQ with n-tier, so I won’t reinvent the wheel. What I will do is talk about strategies on how to leverage LINQ in your Flex app, and how to synchronise data.

Let’s take a think about possible methodologies for sending/receiving related object data from server to client:

  • Safety method: take a little save a little
  • Hungry method: take a little save lots
  • Greedy method: take everything save lots
  • Thrifty method: take everything save a little

I’ll use the following example to illustrate the different methods:

Example Schema

 

Safety Method

The safety method is a simple-to-follow, clear 1:1 system  – load an object when you need it, and load the associated objects as you need them. When you save data back, call the server with either one or a collection of single unrelated objects. You rely on your foreign keys to do all the loading.

All associations within the LINQ to SQL designer (DBML) need to have INTERNAL access, to prevent them being sent to the client, and so that you can still use them within the server code. 

For example: Your client wants the list of projects? It loads them simply via LINQ. Client wants the project’s list of Companies? Load them via the ProjectID. Need to save a Company, send the server the Company object with an associated ProjectID. 

 

Hungry Method

The hungry method involves retrieving just the data objects you require at the time and building up the object model on the client side, eventually sending back an object with related objects. This means you don’t have an excessive number of save calls back to the server, and you don’t rely on the foreign key, you rely on the attached object.

For example:   You may want to load all Projects within the system via LINQ, but don’t want the associated Companies, Employees or Suburbs. You then want to create a new Project and create Companies, Employees. You then call one Save() method to save the new Project, and let LINQ create the entire collection of Company and Employee records.

Now, by default, LINQ will try to send you all the related Companies, Employees and related Suburbs when you ask for the list of Projects.  You have two main options to avoid this:

  1. Change the ACCESS of the association(s) between parent and child within the DBML (LINQ to SQL) editor to INTERNAL. This means only inside the DLL could you access this property.
  2. In your server DLL, NULL any related properties before returning the data to the client. (Literally: project.Companies = null).    

Do you see the problem? Option 1 is the obvious choice but then it means you couldn’t actually send the data back to .NET because if you tried to send a Project back with Companies attached, .NET would say that Project doesn’t have the Companies property as the association as it is internal (and the exception happens during unmarshalling on the server, handled by either the WebORB or FluorineFx DLL).

The Hungry method starts to become a real issue with visibility, and unfortunately, the easiest is to follow option 2 above, which is not a good solution in my opinion.

NOTE: When saving back to the server, do NOT send back both an associated object and an associated foreign key. This will cause .NET to except. EG: If you were to save a Company with an attached Project, you couldn’t also send back Company’s ProjectID (the FK), .NET wouldn’t know which association to use. 

So, in your client code, either remove the Foreign Keys completely from the value object, or use a property getter only to get the foreign key from the association object.

 

Greedy Method

The greedy method is expensive. You are always sending and receiving all (or a lot) of the data in a single call. The advantage is, less trips between the client and server, and minimal client & server code (one Save method to do everything!).

As you can imagine, debugging the Save method can be a headache – as debugging LINQ usually is 🙂 

Now, to Update an object under the greedy method is tricky yet fun. You need to Attach() this new object to the context, along with it’s original from yet another context. 

For example: You want to load all Employees and their Suburb plus their Company and it’s Project. If you wanted to save it back, you’d need to attach it back in. 

Employee SaveEmployee(Employee e) 
{
  DataClassesDataContext context = new DataClassesDataContext();

  //if is Update
  if (e.EmployeeID > 0) 
  {
     //need to attach Employee with a NEW INSTANCE of the DataContext
     Employee oldEmployee = new DataClassesDataContext().Employees.Single( p => p.EmployeeID == e.EmployeeID);

     //now attach it
     context.Attach(e, oldEmployee);
  }
  else
  {
     context.Employees.InsertOnSubmit(e);
  }
  context.SubmitChanges();
  return e;
}

 

This can get even tricker when you’re attaching objects with children which already exist. It can require going through the children and testing their Primary Keys using the method above.

 

Thrify Method

The thrifty method involves receiving all the required data, and just sending back the minimal amount of objects back to the server for a successful save. 

To prevent data being sent back to the server once it is already in the Flex client, requires the use of the Flex metatag: [Transient]. To use Thrifty, we need to rely on Foreign Key associations as LINQ will use the FK on the insert  (just like saving in the Safety method). The advantage is that you have complete control over anything being saved into your database, which can be a real advantage in a intricate & lengthy project. 

Generally using this method, you would need to recall all of your data again, as some associations have been modified. You can try an sychronise these on the client, but it can get VERY tricky. At the very least, you’ll need to return your saved object to get it’s ID if it was just inserted.

For example: You want to load all Projects from the system, but when you save a Project, you don’t t send back it’s Companies, this property is Transient on the client and doesn’t get send back to the server. If you wanted to save a Company, you’d send back a Company object without the associated Project object, but WITH the ProjectID Foreign Key.

 

I personally use a combination of the Thrifty and the Greedy method, deciding when I want to send all the data back, and when I just want to send back an individual object. It still means however, that I need to reload ALL of my data again after a successful save of some object. 

The most important things to remember is attaching to a new context, and not sending both a foreign key and a related foreign object back to the server simultaneously – this upsets LINQ. 

More examples and further investigation to come…





Silverlight vs Flex

16 02 2009

The Silverlight vs Flex argument is hillarious!

We’ve got a statement from Adobe’s CFO here on Silverlight. Personally I think that the CFO wasn’t the best person to comment on Silverlight, but hey, maybe we got an insight into Adobe’s non-censored attitude to Silverlight. 

We’ve got a response from Tim Sneath, a Silverlight evangilist from Microsoft.

We’ve got a debate with an MS evangilist vs an Adobe evangilist. (To be fair to Silverlight, the MS guy seems like he’s a fish out of water. He starts off with a great argument about competition making us better developers, and ends up floundering).

If you ask me, Adobe needs to fix the perception that most developers have that Flash simply sux. 

The problem is, that for years, the Flash platform was only accessible to those freaks who understood and lived in the four dimensional world of Flash. Trying to code on multiple layers with inherited objects in respect to time and still understanding  scope was, and still is, a nightmare

 I think many web developers have poked their noses into Flash – and many developed a natural distaste for it. Those that tended towards design sometimes learned to love it (and we learned to hate them for polluting the web with horrendous Flash websites that were inaccessible and useless). Then came Flex, at a time when web apps and Web 2.0 was the wave of the future. Macromedia released an SDK that finally made the power of Flash available  (“leveragable” in biz speak) to developers who wanted to use it.

Microsoft, on the other hand, have wowed developers ever since they took Java made .NET.  Managed code – with the ever evolving C# – allows us  developers to write pure object oriented code from web sites to windows applications. And thank god. I mean, I don’t have much interest in OS programming or the kernel. I’ll let others specialise in that. I’m interested in engaging, interactive applications, in whichever form they take.

Let’s get back to basics. Adobe specialises in cross platform solutions. Microsoft have a vested interest in Windows. 

Adobe has flash advocates (I’m using this term from now on) who were pre Flex (ewww), and post Flex (like me). Microsoft has .NET evangilists. Actually, I’m a good example, cause I’m both. I trained in Computer Science at university. I worked mainly with Java on Solaris machines. When I finished uni, I wanted to write software that people would use. I started with the web because of it’s reach, and have been focused on it ever since.  So now I write Flex and AIR apps that interface with .NET, and I’m happy to look at other RIA technologies as long as they

  • Compile (I’m sorry, but I like the portability of compiled code); 
  • Allow for OO programming (Yes, AS3 is OO – close enough anyhow); and
  • Are portable; and
  • Are accessible.

I think the issue between Flash & Silverlight is all this use of “Company X” evangilist business. Too many religious connotations. No wonder there’s so much passion in this argument. Everyone’s drawing lines in the sand. WTF? Anyone heard of software architecture? The goal is to understand as many technologies as possible to create the best solution for the client – whoever they may be. The platform is just a means to an end. We’ve got to constantly weigh up the options from all sides to create the right system at the right time.  

I’m giving myself a task. Over the coming months, I’m going to investigate Silverlight further, now that v2 allows me to code the frontend in C#, and because .NET 4.0 is looking to integrate the client and server within the single code base. As I go through it, I’m going to post my findings here, for those who want the quasi-objective truth.





Silverlight Client to Server Remoting

11 02 2009

Being the Flex/AIR to .NET developer that I am, I find Silverlight curious to say the least.

Initially I had assumed that Silverlight 2, as it was deployed inside an ASP.NET side, would automatically handle remote objects from the client to the server. This would alliviate the developer from having to write two classes for her value objects (one on the client and one on the server). Moreover, it would minimise the amount of setup time to get going.

Turns out, it doesn’t. Not in .NET 3.5. You’ve got the options of using Web Services or solutions like WebORB .NET for Silverlight (using AMF).

Posing the question on Twitter today, I was sent the following link from Tim Heuer from the Silverlight team:

http://channel9.msdn.com/pdc2008/PC11/

So, it seems the future of Silverlight, within .NET 4.0 and Visual Studio 2010 is a seamless integration from server to client via the one code base.

Something Mr. Cool (the speaker from said video) didn’t mention in whether or not the server would require the “Dublin” addition to IIS. This seems to be the way .NET 4.0 is achieving the communication from Client to Server. I think I need to test drive VS2010 to be sure.

The code to get the communication between the client and server is fairly bloaty – and seems a bit hackish at this stage. I get the impression MS are trying to quickly increase the functionality of Silverlight before too many people invest in the Flex Framework. MS are down in the fight as Silverlight currently can’t escape the browser unlike AIR  (WPF can too but it’s Windows dependant). I guess they want to entice the developers via integration rather than the software architects by reach.





Flex: Keeping the sort on a datagrid when you change the dataProvider

26 06 2008

Well, there’s a few things you have to dig out a little when it comes to working with the more useful datagrid events.

Say you want to extend the DataGrid into your own component. You want to know when the dataProvider has been changed, and you’d like the new data to be sorted by the previous sort.

The events are as follows:

First, when a dataProvider is changed, the DataGrid instance will fire an mx.events.CollectionEvent.COLLECTION_CHANGE event.

EG:

dataGrid.addEventListener(mx.events.CollectionEvent.COLLECTION_CHANGE, onDataGridDataChange, false, 0, true);

NOTE: Those extra parameters in the attached listener create a “weak reference”. This is to prevent memory leaks.

This signifies that the dataProvider has been changed (should check the “kind” property to ensure it is equal to CollectionEventKind.RESET).

Second, the actual dataProvider itself will fire a COLLECTION_CHANGE itself after it is sorted. In the datagrid listener, you can attach the listener to the dataProvider. When a column heading is clicked, the datagrid will do the sort then call a “refresh()” which will dispatch a CollectionEvent of kind CollectionEventKind.REFRESH.

EG.

private var currentSort:mx.collections.Sort;
private function onDataGridDataChange(evt:CollectionEvent):void
{

                //get the new data collection
                var ac:ArrayCollection = dataGrid.dataProvider as ArrayCollection;
                //attach the collection change event
                ac.addEventListener(CollectionEvent.COLLECTION_CHANGE, onCollectionChanged,false,0,true);

                //if a sort was in place before the data was changed, make sure you apply the sort to this new data
                if (currentSort != null)
                {
                    ac.sort = currentSort;
                    ac.refresh();
                }

}

private function onCollectionChanged(evt:CollectionEvent):void
{

	if (evt.kind == CollectionEventKind.REFRESH)
	{
		var ac:ArrayCollection = evt.currentTarget as ArrayCollection;

		currentSort = ac.sort;

	}

}




The oohs, aahs and waahs of working with the new RIAs

13 06 2008

Just went to an Adobe AIR camp the other day in Sydney.

I must say, the free t-shirts were a great touch. As much as I abhor supporting any company by wearing their brand name,  the triangular AIR logo is pretty nifty.

But I digress.

Even though you can’t look a gift horse in the mouth (or complain about a free lunch), I confess I was more than a trifle disappointed with the whole day.

To summarise, it was a free one day conference put on by Adobe to showcase AIR and relevant technologies.

It was hardly a cheap affair, so I cannot fault Adobe there, and clearly they are trying to introduce AIR as much as possible. Problem is, converts (like me) would have found little to keep them on the edge of their seat. The Adobe evangelists might be great at gathering new sheep, but the rest of the flock are getting restless.

OK, so clearly I’m the one with the problem. But hey, I’m a pretty straight up, garden type variety developer. And I’ve got to some point with Flex and the whole Flash world and gone, hang on, wtf, I need to use Flash Remoting but I don’t understand anything about how it works. Where’s the complete documentation on it? Where are the other developers who know anything about this stuff? I’m crawling through a wasteland of irrelevant Google results.

Imagine if you will:

You’ve started from the ground up, learning Flex. You got a book on Flex, researched it’s capabilities.

You’ve researched various server techniques to enable Flex to handle to goods. Everything out there seems to conclude that flash remoting (using remote objects) is THE solution to remote procedure calls. You’ll end up with cleaner code, and faster service than using Web Services or HTTP Services.

You’ve spent six months developing a sexy little product in flex and using flash remoting to interface with .net. To do this, you’ve used either WebORB or FluorineFx as a solution to handle the remoting (as per the Adobe documentation Flex.NET).

Then you’re going to deploy to Production. Hang on, am I exposing my data here to malicious attacks? Is this a vulnerability? If I deploy as an AIR application, what are the implications then? I want answers, but can’t find them.

I was hoping I’d get some answers at the AIR camp, and maybe I approached it the wrong way – by walking up and hassling Andrew Spaulding whilst he was setting up for his next presentation.  Come to think of it, I couldn’t have done it any worse. But hey, I’m a developer aren’t I?

There was one tidbit that was very interesting from Matt Voermann about skinning up using Flash CS3 – and I will blog it next.





Silverlight vs Flex (or, Why developers hate flash)

25 05 2008

Now *this* is an argument.

I’ve already seen some great points put out in this debate, although I’ve also seen some very indignant and irrational responses to the argument. Microsoft vs Adobe? or, Why do developers hate Flash?

I guess all those hours of frustration sitting in front of Flash (insert version number here) trying to figure out the most inane bug has caused most developers to cast Flash out. (I know I did).

And that by seeing Microsoft’s rather successful business plan of watching and waiting for fantastic ideas to come along before improving them and labelling their own, causes others to vent in hostility  (thanks Apple for Windows, Java for C# & .NET, Adobe for the Expression tools and Silverlight, Google least of all for Maps, etc.)

But if we look at this argument rationally, it’s clear that we developers will benefit from this contest.

Developers should, by their nature, be adaptable.

Instead of jumping the gun to defend our current choice of RIA software, we need to investigate the technologies as they appear and as they are improved. So that when it comes to making the perfect product, we choose the right tools for the job.


Blog Directory - Blogged