Watching file modifications under MacOS. Expert advice needed.

Does anyone has an idea where to start from implementing native library under MacOS that will catch up file
creation/deletion or modification?

This functionality (FileWatcher) is implemented under Windows and greatly speeds up filesystem synchronization and
reduces lockups on focus gain.

Thanks in advance.


--
Maxim Shafirov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
15 comments
Avatar
Permanently deleted user

Hmm,

I guess the people at java-dev@list.apple.com might be helpful.

0
Avatar
Permanently deleted user

Ah,

I just saw that Panther now supports BSD kevent/kqueue. Look for both on the net.

Also there is

http://developer.apple.com/samplecode/Sample_Code/Interapplication_Comm/Folder_Watching.htm

don't know if this is of any help.

0
Avatar
Permanently deleted user

In article <c0odcb$drq$1@is.intellij.net>,
"Maxim Shafirov (JetBrains)" <max@intellij.com> wrote:

Does anyone has an idea where to start from implementing native library under
MacOS that will catch up file
creation/deletion or modification?


Mac OS X 10.3 has a thing called "Folder Actions" which lets you assign
AppleScripts that execute when sppecified events occur on specified
folders:

http://www.apple.com/applescript/folderactions/index.html

But AppleScript may be too slow for your purposes.


Here are some threads from the Cocoa mailing lists that might be useful:

http://cocoa.mamasam.com/COCOADEV/2002/04/1/31203.php
http://cocoa.mamasam.com/COCOADEV/2002/10/2/48644.php
http://cocoa.mamasam.com/COCOADEV/2003/04/2/62224.php


If you want to hire someone to crank it out quickly, you could post on
the mac-dev-jobs mailing list:

http://groups.yahoo.com/group/mac-dev-jobs/


--
Erik Hanson

0
Avatar
Permanently deleted user

In article <c0odcb$drq$1@is.intellij.net>,
"Maxim Shafirov (JetBrains)" <max@intellij.com> wrote:

Does anyone has an idea where to start from implementing native library under
MacOS that will catch up file
creation/deletion or modification?

This functionality (FileWatcher) is implemented under Windows and greatly
speeds up filesystem synchronization and
reduces lockups on focus gain.

Thanks in advance.


This might be useful:



Begin forwarded message:

From: Brian Bergstrand
Date: February 20, 2004 11:52:42 AM PST
Cc: Mac Dev <macosx-dev@omnigroup.com>
Subject: Re: file/directory change events


This is only possible on Panther 10.3.x. For Jaguar you have to poll.

What you want to use is the Kqueue kernel notification support. See
/usr/include/sys/event.h

Here is some test code that I wrote while implementing kqueue support
for the Ext2 driver.

http://cvs.sourceforge.net/viewcvs.py/ext2fsx/test/kqwatch.c?
rev=1.1&view=auto

HTH.


On Feb 20, 2004, at 1:20 PM, Michael Ruflin wrote:

Hi all

I'm trying to write a java library (which includes of course native
code)
that allows listen on directory/file changes on disc (rename, delete,
etc.
as well as changes to the content of a file itself (through writes)).
I already searched the documentation for OsX without luck
(File Manager doesn't seem to provide what I need).

Is there an API that allows listening on files or directory changes?


It may help to know how other OS's do it:

Unix:
For unix exists one library, FAM. FAM uses a pseudo device driver
(imon) to get informations about file changes (or DNotify on newer
linux
systems). If neither of these two exists, it falls back on polling...
Would
it be possible to use this library for OsX too?
http://oss.sgi.com/projects/fam/


Win
On Windows there are two ways, of which one is based on an interesting
idea:
Listeing on file/directory changes by opening a read operation on a
file/directory. A write operation automatically closes the read
operation,
thus indication a change. If there is no API for OsX, would such a way
be possible?

Thanks for reading,

Michael

_______________________________________________
MacOSX-dev mailing list
MacOSX-dev@omnigroup.com
http://www.omnigroup.com/mailman/listinfo/macosx-dev

0

Would this be why I'm getting frequent lockups of Idea on OSX when it gains focus when coming back from sleep? Very annoying. I've already posted it to Apple, but have not yet found the root cause.

0
Avatar
Permanently deleted user

Gregory Pierce wrote:

Would this be why I'm getting frequent lockups

Exactly

--
Maxim Shafirov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0

Any progress on this? I'm a current OS X user that's considering going back to Win32 specifically because of the speed of file operations in OS X compared to Win32 (due to the lack of native support, I'm hoping). I'd love to test something, even if it wasn't very stable, just to see if it's going to make enough of a difference.

Thanks,
Mike

0
Avatar
Permanently deleted user

Michael Werle wrote:

Any progress on this? I'm a current OS X user that's considering going back to Win32 specifically because of the speed of file operations in OS X compared to Win32 (due to the lack of native support, I'm hoping). I'd love to test something, even if it wasn't very stable, just to see if it's going to make enough of a difference.

Thanks,
Mike

Not yet unfortunately. I beg to be the first one who tests new filewatcher on Mac.

--
Maxim Shafirov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Avatar
Permanently deleted user

In case you haven't started yet, someone just released a library for this. Look for "UKKQueue" on:

http://www.zathras.de/programming/cocoa_stuff.php

The author says, "This is a wrapper around kqueue, which is a Unix mechanism that lets you watch for changes made to files. The wrapper class isn't bullet-proof, mind you. It was ripped out of a project I've been working on."

--
Erik Hanson

0
Avatar
Permanently deleted user

Erik Hanson wrote:

In case you haven't started yet, someone just released a library for this. Look for "UKKQueue" on:

http://www.zathras.de/programming/cocoa_stuff.php

The author says, "This is a wrapper around kqueue, which is a Unix mechanism that lets you watch for changes made to files. The wrapper class isn't bullet-proof, mind you. It was ripped out of a project I've been working on."

--
Erik Hanson


The only thing left that prevents us of implementing native filewhatcher under MacOS (Linux has similar problem) is
kqueue only allows to watch for modification on a specified file/folder only (no folder tree recursion) and requires to
open a file handler per every watched directory. As we all know allowed file handlers per process is not unlimited and
certainly will not carry out 10K+ folders of a relatively big project.

Any further ideas everyone?

--
Maxim Shafirov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0
Avatar
Permanently deleted user

Are you sure that watching each folder won't work for thousands of folders? I can't imagine these things were designed without being able to do something as simple as watching a folder recursively.

0
Avatar
Permanently deleted user

Did you guys take a look at FAM[/url]? If this is not exactly what you're looking for maybe it's a starting point for your own code.

0
Avatar
Permanently deleted user

FAM faq:

"We develop primarily on Red Hat Linux and IRIX, so you should have no problems running it on one of those operating systems. FAM also runs on most GNU/Linux and BSD distributions, and should be fairly easy to port to other Unix-like operating systems as FAM uses GNU autotools."

BSD and MacOS X share a lot in common, maybe this would work?

0
Avatar
Permanently deleted user

andrew wulf wrote:

FAM faq:

"We develop primarily on Red Hat Linux and IRIX, so you should have no problems running it on one of those operating systems. FAM also runs on most GNU/Linux and BSD distributions, and should be fairly easy to port to other Unix-like operating systems as FAM uses GNU autotools."

BSD and MacOS X share a lot in common, maybe this would work?

FAM uses just the same scheme i.e. watching every single dir unfortunately.

--
Maxim Shafirov
IntelliJ Labs / JetBrains Inc.
http://www.intellij.com
"Develop with pleasure!"

0

My I suggest that you implement the naive solution (watching each dir with a, potentially, horrendous number of file handles) in the short term and make it an option that can be turned off so those who cannot change their OS to handle this will still be able to use IDEA?

I'm on a relatively small (<500 classes) embedded project, and due to various reasons, probably about a third of those classes are generated, meaning that we have ant scripts that rebuild 10 or 20 at a time. Every time this happens, IDEA stops for 15-60 seconds before it even starts parsing them, which is really annoying.

Because we're a startup, we're developing on relatively limited machines (12" PowerBook G4's). I love using IDEA, but the team I'm on is very reluctant to switch back to PC's and the lack of native file watching is making the use of IDEA so painful that I'm near to switching to emacs, as sad as that is.

Granted, the naive solution has lots of problems, but it would be of great help to many in the short term, so long as it could be turned off if there was a big problem.

--Mike

0

Please sign in to leave a comment.