scala.io Path/File/Directory hierarchy

On Sat, Sep 05, 2009 at 08:35:32PM -0400, Erik Engbrecht wrote:
> The downsides of this structure are that (1) it's more complicated

Only a little.

> and (2) the underlying file system is quite mutable so a Directory
> object that corresponds to a directory on the file system one minute
> could correspond to a file on the file system the next.

Indeed, and this haunts any attempt to impose firm divisions. Writing
firmly structured immutable in-memory objects which represent filesystem
entities which can change at any time is like producing top quality
ammunition for your jello gun. I'll take a look at your hierarchy.

On Sun, Sep 6, 2009 at 3:53 PM, Paul Phillips wrote:
> Indeed, and this haunts any attempt to impose firm divisions.  Writing
> firmly structured immutable in-memory objects which represent filesystem
> entities which can change at any time is like producing top quality
> ammunition for your jello gun.  I'll take a look at your hierarchy.

Maybe some ideas can be borrowed from Apple?
http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/7

Apparently they are moving in the direction of having URI represent
anything path related.

In any case, just a thought about files, directories and paths.
Another way to look at it is to have a Path (or Location, or URI) be
the reference to the mutable filesystem object (if it exsits) and then
have fileData:Option[FileData] and dirData:Option[FileData] as query
methods on this object returning immutable metadata records about the
resource (as of the time when the query methods was called).

BR,
John

I think that there is no need to have Directory class at all. All
"directory" method can be added to File class (as in java.io.File).
java.io.File exists for 10 years, it has problems, but developers
rarely complained about absence of Directory class.

Subclassing has downsides, as mentioned before:

— same path on filesystem may be file or directory at different time,
or may change existence
— file type check maybe impossible because of lack of permissions
— permature checking of file type is expensive
— multiple classes instead of one make API harder to understand

S.

On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht wrote:
> Hi all,
>   I took a look at some of Paul's recent commits with IO and have some
> comments regarding the overall structure.
> It basically look like this:
> Path
>     File
>     Directory
> All of the above are concrete classes.  Path represents an abstract pathname
> is is very close in semantics to java.io.File, which quite misleadingly can
> be a file, directory, or none-of-the-above because it doesn't exist on the
> filesystem.  File and Directory are specializations of Path that represent
> files and directories respectively, whether they exist on the filesystem or
> not.
> The problem is that Path has many methods that apply to files or directories
> (more so to directories) rather than methods that apply to both.  I think
> this is appropriate, given what Path is supposed to represent, but the
> existing hierarchy leads to File inheriting a bunch of methods that really
> only make sense for working with directories.
> I'd like to propose an alternate hierarchy:
> Location
>     Path
>     File
>     Directory
> Where location is abstract and Path has something close to the union of the
> methods of File and Directory.  The definition of the methods should be
> factored out into mixins, such as follows:
> trait FileOps extends Location         // operations that make sense only on
> files
> trait DirectoryOps extends Location //operations that make sense only on
> directories
> abstract class Location
> class File extends Location with FileOps
> class Directory extends Location with DirectoryOps
> class Path extends Location with FileOps with DirectoryOps
> A more-or-less complete implementation using this pattern can be found here:
> http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0...
> The downsides of this structure are that (1) it's more complicated, and (2)
> the underlying file system is quite mutable so a Directory object that
> corresponds to a directory on the file system one minute could correspond to
> a file on the file system the next.
> What do you guys think?  Given that I already have something that follows
> the structure I'm proposing, I could put together a patch relatively
> quickly.
> -Erik
>
>

When you use what you expect to be a regular file, why would you want
to see .listFiles or other directory-specific methods?

I'd go for File and Directory and not try to enforce that each
actually refers to a physical regular file or directory.

2009/9/6 Stepan Koltsov :
> I think that there is no need to have Directory class at all. All
> "directory" method can be added to File class (as in java.io.File).
> java.io.File exists for 10 years, it has problems, but developers
> rarely complained about absence of Directory class.
>
> Subclassing has downsides, as mentioned before:
>
> — same path on filesystem may be file or directory at different time,
> or may change existence
> — file type check maybe impossible because of lack of permissions
> — permature checking of file type is expensive
> — multiple classes instead of one make API harder to understand
>
> S.
>
>
> On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht wrote:
>> Hi all,
>>   I took a look at some of Paul's recent commits with IO and have some
>> comments regarding the overall structure.
>> It basically look like this:
>> Path
>>     File
>>     Directory
>> All of the above are concrete classes.  Path represents an abstract pathname
>> is is very close in semantics to java.io.File, which quite misleadingly can
>> be a file, directory, or none-of-the-above because it doesn't exist on the
>> filesystem.  File and Directory are specializations of Path that represent
>> files and directories respectively, whether they exist on the filesystem or
>> not.
>> The problem is that Path has many methods that apply to files or directories
>> (more so to directories) rather than methods that apply to both.  I think
>> this is appropriate, given what Path is supposed to represent, but the
>> existing hierarchy leads to File inheriting a bunch of methods that really
>> only make sense for working with directories.
>> I'd like to propose an alternate hierarchy:
>> Location
>>     Path
>>     File
>>     Directory
>> Where location is abstract and Path has something close to the union of the
>> methods of File and Directory.  The definition of the methods should be
>> factored out into mixins, such as follows:
>> trait FileOps extends Location         // operations that make sense only on
>> files
>> trait DirectoryOps extends Location //operations that make sense only on
>> directories
>> abstract class Location
>> class File extends Location with FileOps
>> class Directory extends Location with DirectoryOps
>> class Path extends Location with FileOps with DirectoryOps
>> A more-or-less complete implementation using this pattern can be found here:
>> http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0...
>> The downsides of this structure are that (1) it's more complicated, and (2)
>> the underlying file system is quite mutable so a Directory object that
>> corresponds to a directory on the file system one minute could correspond to
>> a file on the file system the next.
>> What do you guys think?  Given that I already have something that follows
>> the structure I'm proposing, I could put together a patch relatively
>> quickly.
>> -Erik
>>
>>
>

I sincerely hope that someone is taking a close look at the new
filesystem interface that's coming out of NIO2 and will be included in
JDK 7,

http://java.sun.com/developer/technicalArticles/javase/nio/

Cheers,

Miles

On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson wrote:
> When you use what you expect to be a regular file, why would you want
> to see .listFiles or other directory-specific methods?

Let, we have Directory subclass. We've got Directory instance. Then
we've deleted directory in the filesystem. We still have Directory
instance. Why would you want to see listFiles method on this directory
instance?

Object under File (i. e. filesystem path) is mutable by nature. This
is why having directory methods on file object is OK to me.

> I'd go for File and Directory and not try to enforce that each
> actually refers to a physical regular file or directory.

Will listFiles method return mix of File and Directory in your proposal?

S.

> 2009/9/6 Stepan Koltsov :
>> I think that there is no need to have Directory class at all. All
>> "directory" method can be added to File class (as in java.io.File).
>> java.io.File exists for 10 years, it has problems, but developers
>> rarely complained about absence of Directory class.
>>
>> Subclassing has downsides, as mentioned before:
>>
>> — same path on filesystem may be file or directory at different time,
>> or may change existence
>> — file type check maybe impossible because of lack of permissions
>> — permature checking of file type is expensive
>> — multiple classes instead of one make API harder to understand
>>
>> S.
>>
>>
>> On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht wrote:
>>> Hi all,
>>>   I took a look at some of Paul's recent commits with IO and have some
>>> comments regarding the overall structure.
>>> It basically look like this:
>>> Path
>>>     File
>>>     Directory
>>> All of the above are concrete classes.  Path represents an abstract pathname
>>> is is very close in semantics to java.io.File, which quite misleadingly can
>>> be a file, directory, or none-of-the-above because it doesn't exist on the
>>> filesystem.  File and Directory are specializations of Path that represent
>>> files and directories respectively, whether they exist on the filesystem or
>>> not.
>>> The problem is that Path has many methods that apply to files or directories
>>> (more so to directories) rather than methods that apply to both.  I think
>>> this is appropriate, given what Path is supposed to represent, but the
>>> existing hierarchy leads to File inheriting a bunch of methods that really
>>> only make sense for working with directories.
>>> I'd like to propose an alternate hierarchy:
>>> Location
>>>     Path
>>>     File
>>>     Directory
>>> Where location is abstract and Path has something close to the union of the
>>> methods of File and Directory.  The definition of the methods should be
>>> factored out into mixins, such as follows:
>>> trait FileOps extends Location         // operations that make sense only on
>>> files
>>> trait DirectoryOps extends Location //operations that make sense only on
>>> directories
>>> abstract class Location
>>> class File extends Location with FileOps
>>> class Directory extends Location with DirectoryOps
>>> class Path extends Location with FileOps with DirectoryOps
>>> A more-or-less complete implementation using this pattern can be found here:
>>> http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0...
>>> The downsides of this structure are that (1) it's more complicated, and (2)
>>> the underlying file system is quite mutable so a Directory object that
>>> corresponds to a directory on the file system one minute could correspond to
>>> a file on the file system the next.
>>> What do you guys think?  Given that I already have something that follows
>>> the structure I'm proposing, I could put together a patch relatively
>>> quickly.
>>> -Erik
>>>
>>>
>>
>
>
>
> --
> Ricky Clarkson
> Java Programmer, AD Holdings
> +44 1565 770804
> Skype: ricky_clarkson
> Google Talk: ricky.clarkson@gmail.com
>

On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson wrote:
> When you use what you expect to be a regular file, why would you want
> to see .listFiles or other directory-specific methods?
>
> I'd go for File and Directory and not try to enforce that each
> actually refers to a physical regular file or directory.

Also, except files and directories there are also nonexistent files
and symlinks (not counting devices and named sockets and pipes).

If it is wrong to have listFiles on File object, then it is wrong to
have length() method on nonexistent file. So there must exists Path
base class for File and Directory. And also, why having ctime() method
on nonexistend object? So there must exists ExistingPath, subclass of
Path, and base class of File and Directory.

Look at typical code that deletes old files and directories.

Old, java style:

===
def deleteOldFiles(dir: File) =
for (file <- dir.list())
if (file.isRegular() && file.ctime() < now - 3600)
file.delete();
===

Proposed:

===
def deleteOldFiles(dir: Directory) =
for (path <- dir.list())
file match {
case f: File =>
if (file.ctime() < now - 3600)
file.delete();
case _ =>
}
===

Old style is singificantly shorter and easier.

I have to say one more thing. File is not a representation of "regular
file" or filesystem object. File represents a path on the filesystem.
Filesystem has "list children" operation. This operation is
represented by "list()" method of File object, that is interface to
the filesystem.

S.

trait Location {
  def name: String
  def path: String
  // other abstract common methods here
}

object File {
  def apply(jFile: JFile): File = new Path(jFile)
  // other convenient factory methods
}

trait File extends Location {
  // abstract file methods here
}

object Directory {
  def apply(jFile: JFile): Directory = new Path(jFile)
  def home: Directory = // ..
  def current: Directory = // ..
  def temp: Directory = // ..
}

trait Directory extends Location {
  // abstract directory methods here
}

class Path(val jFile: JFile) extends File with Directory {

  // full implementation here

}

I think that there is no need to have Directory class at all. All
"directory" method can be added to File class (as in java.io.File).
java.io.File exists for 10 years, it has problems, but developers
rarely complained about absence of Directory class.

Subclassing has downsides, as mentioned before:

— same path on filesystem may be file or directory at different time,
or may change existence
— file type check maybe impossible because of lack of permissions
— permature checking of file type is expensive
— multiple classes instead of one make API harder to understand

S.


On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> Hi all,
>   I took a look at some of Paul's recent commits with IO and have some
> comments regarding the overall structure.
> It basically look like this:
> Path
>     File
>     Directory
> All of the above are concrete classes.  Path represents an abstract pathname
> is is very close in semantics to java.io.File, which quite misleadingly can
> be a file, directory, or none-of-the-above because it doesn't exist on the
> filesystem.  File and Directory are specializations of Path that represent
> files and directories respectively, whether they exist on the filesystem or
> not.
> The problem is that Path has many methods that apply to files or directories
> (more so to directories) rather than methods that apply to both.  I think
> this is appropriate, given what Path is supposed to represent, but the
> existing hierarchy leads to File inheriting a bunch of methods that really
> only make sense for working with directories.
> I'd like to propose an alternate hierarchy:
> Location
>     Path
>     File
>     Directory
> Where location is abstract and Path has something close to the union of the
> methods of File and Directory.  The definition of the methods should be
> factored out into mixins, such as follows:
> trait FileOps extends Location         // operations that make sense only on
> files
> trait DirectoryOps extends Location //operations that make sense only on
> directories
> abstract class Location
> class File extends Location with FileOps
> class Directory extends Location with DirectoryOps
> class Path extends Location with FileOps with DirectoryOps
> A more-or-less complete implementation using this pattern can be found here:
> http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0eb134f5857/src/main/scala/scalax/io
> The downsides of this structure are that (1) it's more complicated, and (2)
> the underlying file system is quite mutable so a Directory object that
> corresponds to a directory on the file system one minute could correspond to
> a file on the file system the next.
> What do you guys think?  Given that I already have something that follows
> the structure I'm proposing, I could put together a patch relatively
> quickly.
> -Erik
>
>

On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson<ricky.clarkson@gmail.com> wrote:
> When you use what you expect to be a regular file, why would you want
> to see .listFiles or other directory-specific methods?
>
> I'd go for File and Directory and not try to enforce that each
> actually refers to a physical regular file or directory.

Also, except files and directories there are also nonexistent files
and symlinks (not counting devices and named sockets and pipes).

If it is wrong to have listFiles on File object, then it is wrong to
have length() method on nonexistent file. So there must exists Path
base class for File and Directory. And also, why having ctime() method
on nonexistend object? So there must exists ExistingPath, subclass of
Path, and base class of File and Directory.


Look at typical code that deletes old files and directories.

Old, java style:

===
def deleteOldFiles(dir: File) =
 for (file <- dir.list())
   if (file.isRegular() && file.ctime() < now - 3600)
     file.delete();
===

Proposed:

===
def deleteOldFiles(dir: Directory) =
 for (path <- dir.list())
   file match {
     case f: File =>
       if (file.ctime() < now - 3600)
         file.delete();
     case _ =>
   }
===

Old style is singificantly shorter and easier.

I have to say one more thing. File is not a representation of "regular
file" or filesystem object. File represents a path on the filesystem.
Filesystem has "list children" operation. This operation is
represented by "list()" method of File object, that is interface to
the filesystem.

S.

I'd switch Path and File (Path is a base class for everything, File
inherits everything), and introduce RegularFile:

===
class Path {
def exists() = ...
}

trait RegularFile extends Path {
...
}

trait Directory extends Path {
...
def list(): Seq[File] = ... // <- returns instances of File, agreed?
}

class File extends Path with RegularFile with Directory
===

And also, Location is redundant.

S.

On Sun, Sep 6, 2009 at 22:23, Erik Engbrecht wrote:
> Stepan,
> I think having multiple types to represent different concepts would make the
> API easier to understand, but the others are valid points.  The inherent
> mutability of the filesystem is a giant elephant walking around the room.
> I've been toying around with different approaches, and discussing it will
> Paul and Ishmael on IRC.  My current thinking leans towards something like
> the following.
>
> trait Location {
> def name: String
> def path: String
> // other abstract common methods here
> }
>
> object File {
> def apply(jFile: JFile): File = new Path(jFile)
> // other convenient factory methods
> }
>
> trait File extends Location {
> // abstract file methods here
> }
>
> object Directory {
> def apply(jFile: JFile): Directory = new Path(jFile)
> def home: Directory = // ..
> def current: Directory = // ..
> def temp: Directory = // ..
> }
>
> trait Directory extends Location {
> // abstract directory methods here
> }
>
> class Path(val jFile: JFile) extends File with Directory {
>
> // full implementation here
>
> }
>
> In this scheme all the implementation in really on the Path class which is
> more-or-less a Scala-ized variant of java.io.File.  Directory and File
> provide marker interfaces to distinguish what the underlying object on the
> filesystem is expected to be.  Performance would be equivalent to just
> having Path, and if the programmer wants to work in Paths and call methods
> on require a File or Directory he can without performing any conversions.
>  His intent just wouldn't be as clearly shown in the code.
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik
>
>
> On Sun, Sep 6, 2009 at 1:38 PM, Stepan Koltsov wrote:
>>
>> I think that there is no need to have Directory class at all. All
>> "directory" method can be added to File class (as in java.io.File).
>> java.io.File exists for 10 years, it has problems, but developers
>> rarely complained about absence of Directory class.
>>
>> Subclassing has downsides, as mentioned before:
>>
>> — same path on filesystem may be file or directory at different time,
>> or may change existence
>> — file type check maybe impossible because of lack of permissions
>> — permature checking of file type is expensive
>> — multiple classes instead of one make API harder to understand
>>
>> S.
>>
>>
>> On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht
>> wrote:
>> > Hi all,
>> >   I took a look at some of Paul's recent commits with IO and have some
>> > comments regarding the overall structure.
>> > It basically look like this:
>> > Path
>> >     File
>> >     Directory
>> > All of the above are concrete classes.  Path represents an abstract
>> > pathname
>> > is is very close in semantics to java.io.File, which quite misleadingly
>> > can
>> > be a file, directory, or none-of-the-above because it doesn't exist on
>> > the
>> > filesystem.  File and Directory are specializations of Path that
>> > represent
>> > files and directories respectively, whether they exist on the filesystem
>> > or
>> > not.
>> > The problem is that Path has many methods that apply to files or
>> > directories
>> > (more so to directories) rather than methods that apply to both.  I
>> > think
>> > this is appropriate, given what Path is supposed to represent, but the
>> > existing hierarchy leads to File inheriting a bunch of methods that
>> > really
>> > only make sense for working with directories.
>> > I'd like to propose an alternate hierarchy:
>> > Location
>> >     Path
>> >     File
>> >     Directory
>> > Where location is abstract and Path has something close to the union of
>> > the
>> > methods of File and Directory.  The definition of the methods should be
>> > factored out into mixins, such as follows:
>> > trait FileOps extends Location         // operations that make sense
>> > only on
>> > files
>> > trait DirectoryOps extends Location //operations that make sense only on
>> > directories
>> > abstract class Location
>> > class File extends Location with FileOps
>> > class Directory extends Location with DirectoryOps
>> > class Path extends Location with FileOps with DirectoryOps
>> > A more-or-less complete implementation using this pattern can be found
>> > here:
>> >
>> > http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0...
>> > The downsides of this structure are that (1) it's more complicated, and
>> > (2)
>> > the underlying file system is quite mutable so a Directory object that
>> > corresponds to a directory on the file system one minute could
>> > correspond to
>> > a file on the file system the next.
>> > What do you guys think?  Given that I already have something that
>> > follows
>> > the structure I'm proposing, I could put together a patch relatively
>> > quickly.
>> > -Erik
>> >
>> >
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On Sun, Sep 6, 2009 at 8:23 PM, Erik Engbrecht wrote:
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik

IIRC from the discussions of including Reiser4 in Linux it is
perfectly valid for an object in the Linux VFS to be both a file (byte
stream) and a directory (collection of files) at the same time.

BR,
John

I'd switch Path and File (Path is a base class for everything, File
inherits everything), and introduce RegularFile:

===
class Path {
 def exists() = ...
}

trait RegularFile extends Path {
 ...
}

trait Directory extends Path {
 ...
 def list(): Seq[File] = ... // <- returns instances of File, agreed?
}

class File extends Path with RegularFile with Directory
===

And also, Location is redundant.

S.


On Sun, Sep 6, 2009 at 22:23, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> Stepan,
> I think having multiple types to represent different concepts would make the
> API easier to understand, but the others are valid points.  The inherent
> mutability of the filesystem is a giant elephant walking around the room.
> I've been toying around with different approaches, and discussing it will
> Paul and Ishmael on IRC.  My current thinking leans towards something like
> the following.
>
> trait Location {
>   def name: String
>   def path: String
>   // other abstract common methods here
> }
>
> object File {
>   def apply(jFile: JFile): File = new Path(jFile)
>   // other convenient factory methods
> }
>
> trait File extends Location {
>   // abstract file methods here
> }
>
> object Directory {
>   def apply(jFile: JFile): Directory = new Path(jFile)
>   def home: Directory = // ..
>   def current: Directory = // ..
>   def temp: Directory = // ..
> }
>
> trait Directory extends Location {
>   // abstract directory methods here
> }
>
> class Path(val jFile: JFile) extends File with Directory {
>
>   // full implementation here
>
> }
>
> In this scheme all the implementation in really on the Path class which is
> more-or-less a Scala-ized variant of java.io.File.  Directory and File
> provide marker interfaces to distinguish what the underlying object on the
> filesystem is expected to be.  Performance would be equivalent to just
> having Path, and if the programmer wants to work in Paths and call methods
> on require a File or Directory he can without performing any conversions.
>  His intent just wouldn't be as clearly shown in the code.
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik
>
>
> On Sun, Sep 6, 2009 at 1:38 PM, Stepan Koltsov <stepancheg@mx1.ru> wrote:
>>
>> I think that there is no need to have Directory class at all. All
>> "directory" method can be added to File class (as in java.io.File).
>> java.io.File exists for 10 years, it has problems, but developers
>> rarely complained about absence of Directory class.
>>
>> Subclassing has downsides, as mentioned before:
>>
>> — same path on filesystem may be file or directory at different time,
>> or may change existence
>> — file type check maybe impossible because of lack of permissions
>> — permature checking of file type is expensive
>> — multiple classes instead of one make API harder to understand
>>
>> S.
>>
>>
>> On Sun, Sep 6, 2009 at 04:35, Erik Engbrecht<erik.engbrecht@gmail.com>
>> wrote:
>> > Hi all,
>> >   I took a look at some of Paul's recent commits with IO and have some
>> > comments regarding the overall structure.
>> > It basically look like this:
>> > Path
>> >     File
>> >     Directory
>> > All of the above are concrete classes.  Path represents an abstract
>> > pathname
>> > is is very close in semantics to java.io.File, which quite misleadingly
>> > can
>> > be a file, directory, or none-of-the-above because it doesn't exist on
>> > the
>> > filesystem.  File and Directory are specializations of Path that
>> > represent
>> > files and directories respectively, whether they exist on the filesystem
>> > or
>> > not.
>> > The problem is that Path has many methods that apply to files or
>> > directories
>> > (more so to directories) rather than methods that apply to both.  I
>> > think
>> > this is appropriate, given what Path is supposed to represent, but the
>> > existing hierarchy leads to File inheriting a bunch of methods that
>> > really
>> > only make sense for working with directories.
>> > I'd like to propose an alternate hierarchy:
>> > Location
>> >     Path
>> >     File
>> >     Directory
>> > Where location is abstract and Path has something close to the union of
>> > the
>> > methods of File and Directory.  The definition of the methods should be
>> > factored out into mixins, such as follows:
>> > trait FileOps extends Location         // operations that make sense
>> > only on
>> > files
>> > trait DirectoryOps extends Location //operations that make sense only on
>> > directories
>> > abstract class Location
>> > class File extends Location with FileOps
>> > class Directory extends Location with DirectoryOps
>> > class Path extends Location with FileOps with DirectoryOps
>> > A more-or-less complete implementation using this pattern can be found
>> > here:
>> >
>> > http://github.com/eengbrec/Scalax.IO/tree/2d9e0feb15d77856760b8e9d82cfd0eb134f5857/src/main/scala/scalax/io
>> > The downsides of this structure are that (1) it's more complicated, and
>> > (2)
>> > the underlying file system is quite mutable so a Directory object that
>> > corresponds to a directory on the file system one minute could
>> > correspond to
>> > a file on the file system the next.
>> > What do you guys think?  Given that I already have something that
>> > follows
>> > the structure I'm proposing, I could put together a patch relatively
>> > quickly.
>> > -Erik
>> >
>> >
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On Sun, Sep 6, 2009 at 8:23 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik

IIRC from the discussions of including Reiser4 in Linux it is
perfectly valid for an object in the Linux VFS to be both a file (byte
stream) and a directory (collection of files) at the same time.

BR,
John

Directories are not the only containers.
Under windows we can have .lnk files that link to directories, there's also the OLE container format, not to mention variations on compression (zip, rar, etc...).  The idea that a file can also be a directory has been mention for reiser4, and a similar concept also exists in the form of alternate streams for HPFS and NTFS.  I'm sure that there are other edge-cases.
There are also things that are neither files nor containers.  Named pipes have already been mentioned, and symlinks can be viewed as symlinks or simply be followed to the linked object.


All we really can be sure of is that we have a local handle of some sort, possibly to be represented as a subset of URLs (which, in turn, is a subset of URIs).  Within the run time of a system, that handle can be used in different roles.

Perhaps this calls for some form of dynamic composition.  As with the GoF flyweight, roles, or the ability to apply traits to existing object instances.  I think we have here a perfect scenario here for application of the DCI (Data/Context/Interaction) pattern.


On Sun, Sep 6, 2009 at 7:38 PM, John Nilsson <john@milsson.nu> wrote:

On Sun, Sep 6, 2009 at 8:23 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik

IIRC from the discussions of including Reiser4 in Linux it is
perfectly valid for an object in the Linux VFS to be both a file (byte
stream) and a directory (collection of files) at the same time.

BR,
John

...and the plot thickens....
My feeling is that once we have some nice friendly abstractions for "normal" filesystem stuff that doesn't leak like a sieve then we can start generalizing it over other filesystem-like abstractions (archive files, URLs, etc).
So we start with the normal filesystem stuff, then refactor it for archive files, then again for URL and/or URIs.
But that's just the direction I'm headed in.  Others are more than welcome to investigate from alternative angles.

On Sun, Sep 6, 2009 at 2:57 PM, Kevin Wright <kev.lee.wright@googlemail.com> wrote:

Directories are not the only containers.
Under windows we can have .lnk files that link to directories, there's also the OLE container format, not to mention variations on compression (zip, rar, etc...).  The idea that a file can also be a directory has been mention for reiser4, and a similar concept also exists in the form of alternate streams for HPFS and NTFS.  I'm sure that there are other edge-cases.
There are also things that are neither files nor containers.  Named pipes have already been mentioned, and symlinks can be viewed as symlinks or simply be followed to the linked object.


All we really can be sure of is that we have a local handle of some sort, possibly to be represented as a subset of URLs (which, in turn, is a subset of URIs).  Within the run time of a system, that handle can be used in different roles.

Perhaps this calls for some form of dynamic composition.  As with the GoF flyweight, roles, or the ability to apply traits to existing object instances.  I think we have here a perfect scenario here for application of the DCI (Data/Context/Interaction) pattern.


On Sun, Sep 6, 2009 at 7:38 PM, John Nilsson <john@milsson.nu> wrote:

On Sun, Sep 6, 2009 at 8:23 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> My problem with this is that a Path isn't really both a File and a
> Directory, it is a File or a Directory or it doesn't exist and could be
> created as either on the filesystem (assuming permissions or other factors
> won't prevent it), so I feel like the type signature is deceiving.  Another,
> perhaps worse, problem is that this scheme loses the isValid() (or something
> like that) call that Paul currently has on File and Directory in scala.io.
> -Erik

IIRC from the discussions of including Reiser4 in Linux it is
perfectly valid for an object in the Linux VFS to be both a file (byte
stream) and a directory (collection of files) at the same time.

BR,
John

--
http://erikengbrecht.blogspot.com/

On Sun, Sep 6, 2009 at 22:32, Erik Engbrecht wrote:
> Based on what we currently has in scalax.io on GitHub, you example wouldn't
> be any longer with the separation.  It would actually look like this:
> def deleteOldFile(dir: Directory) {
>   for(file <- dir.files if file.lastModified < (now - 3600)) {
>     file.delete()
>   }
> }

OK, scalax.io does not work here:

===
def deleteTrash(dir: File)
for (file <- dir.list())
if (file.isSymlink || (file.isRegular && file.lastModified < now - 3600)))
file.delete()
===

S.

> On Sun, Sep 6, 2009 at 2:21 PM, Stepan Koltsov wrote:
>>
>> On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson
>> wrote:
>> > When you use what you expect to be a regular file, why would you want
>> > to see .listFiles or other directory-specific methods?
>> >
>> > I'd go for File and Directory and not try to enforce that each
>> > actually refers to a physical regular file or directory.
>>
>> Also, except files and directories there are also nonexistent files
>> and symlinks (not counting devices and named sockets and pipes).
>>
>> If it is wrong to have listFiles on File object, then it is wrong to
>> have length() method on nonexistent file. So there must exists Path
>> base class for File and Directory. And also, why having ctime() method
>> on nonexistend object? So there must exists ExistingPath, subclass of
>> Path, and base class of File and Directory.
>>
>>
>> Look at typical code that deletes old files and directories.
>>
>> Old, java style:
>>
>> ===
>> def deleteOldFiles(dir: File) =
>>  for (file <- dir.list())
>>    if (file.isRegular() && file.ctime() < now - 3600)
>>      file.delete();
>> ===
>>
>> Proposed:
>>
>> ===
>> def deleteOldFiles(dir: Directory) =
>>  for (path <- dir.list())
>>    file match {
>>      case f: File =>
>>        if (file.ctime() < now - 3600)
>>          file.delete();
>>      case _ =>
>>    }
>> ===
>>
>> Old style is singificantly shorter and easier.
>>
>> I have to say one more thing. File is not a representation of "regular
>> file" or filesystem object. File represents a path on the filesystem.
>> Filesystem has "list children" operation. This operation is
>> represented by "list()" method of File object, that is interface to
>> the filesystem.
>>
>> S.
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On Sun, Sep 6, 2009 at 22:32, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> Based on what we currently has in scalax.io on GitHub, you example wouldn't
> be any longer with the separation.  It would actually look like this:
> def deleteOldFile(dir: Directory) {
>   for(file <- dir.files if file.lastModified < (now - 3600)) {
>     file.delete()
>   }
> }

OK, scalax.io does not work here:

===
def deleteTrash(dir: File)
 for (file <- dir.list())
   if (file.isSymlink || (file.isRegular && file.lastModified < now - 3600)))
     file.delete()
===

S.


> On Sun, Sep 6, 2009 at 2:21 PM, Stepan Koltsov <stepancheg@mx1.ru> wrote:
>>
>> On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson<ricky.clarkson@gmail.com>
>> wrote:
>> > When you use what you expect to be a regular file, why would you want
>> > to see .listFiles or other directory-specific methods?
>> >
>> > I'd go for File and Directory and not try to enforce that each
>> > actually refers to a physical regular file or directory.
>>
>> Also, except files and directories there are also nonexistent files
>> and symlinks (not counting devices and named sockets and pipes).
>>
>> If it is wrong to have listFiles on File object, then it is wrong to
>> have length() method on nonexistent file. So there must exists Path
>> base class for File and Directory. And also, why having ctime() method
>> on nonexistend object? So there must exists ExistingPath, subclass of
>> Path, and base class of File and Directory.
>>
>>
>> Look at typical code that deletes old files and directories.
>>
>> Old, java style:
>>
>> ===
>> def deleteOldFiles(dir: File) =
>>  for (file <- dir.list())
>>    if (file.isRegular() && file.ctime() < now - 3600)
>>      file.delete();
>> ===
>>
>> Proposed:
>>
>> ===
>> def deleteOldFiles(dir: Directory) =
>>  for (path <- dir.list())
>>    file match {
>>      case f: File =>
>>        if (file.ctime() < now - 3600)
>>          file.delete();
>>      case _ =>
>>    }
>> ===
>>
>> Old style is singificantly shorter and easier.
>>
>> I have to say one more thing. File is not a representation of "regular
>> file" or filesystem object. File represents a path on the filesystem.
>> Filesystem has "list children" operation. This operation is
>> represented by "list()" method of File object, that is interface to
>> the filesystem.
>>
>> S.
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On Mon, Sep 7, 2009 at 4:56 PM, Erik Engbrecht wrote:
> http://github.com/eengbrec/Scalax.IO/issues/#issue/2
> There is no isSymlink function (there isn't on on java.io.File, either), but

No, but there is on java.nio.file.Path,

http://download.java.net/jdk7/docs/api/java/nio/file/Path.html

If you haven't already taken a very close look at this I suggest you do now.

Cheers,

Miles

There are ways to test if file is symlink, for example,

===
boolean isSymlink(File f) {
File cp = file.getParentFile().getCanonicalFile();
File f1 = new File(cp, f.getName());
return !f1.equals(f1.getCanonicalFile());
}
===

or by using JNI, or by invoking external stat command, or using NIO.2.

S.

On Mon, Sep 7, 2009 at 19:56, Erik Engbrecht wrote:
> Noted.
> http://github.com/eengbrec/Scalax.IO/issues/#issue/2
> There is no isSymlink function (there isn't on on java.io.File, either), but
> I think it could be implemented by comparing the canonical path to the
> absolute path and accounting for the fact that "." and ".." occurrences are
> removed when converting to canonical, not absolute.
>
> On Mon, Sep 7, 2009 at 11:20 AM, Stepan Koltsov wrote:
>>
>> On Sun, Sep 6, 2009 at 22:32, Erik Engbrecht
>> wrote:
>> > Based on what we currently has in scalax.io on GitHub, you example
>> > wouldn't
>> > be any longer with the separation.  It would actually look like this:
>> > def deleteOldFile(dir: Directory) {
>> >   for(file <- dir.files if file.lastModified < (now - 3600)) {
>> >     file.delete()
>> >   }
>> > }
>>
>> OK, scalax.io does not work here:
>>
>> ===
>> def deleteTrash(dir: File)
>>  for (file <- dir.list())
>>    if (file.isSymlink || (file.isRegular && file.lastModified < now -
>> 3600)))
>>      file.delete()
>> ===
>>
>> S.
>>
>>
>> > On Sun, Sep 6, 2009 at 2:21 PM, Stepan Koltsov
>> > wrote:
>> >>
>> >> On Sun, Sep 6, 2009 at 21:47, Ricky Clarkson
>> >> wrote:
>> >> > When you use what you expect to be a regular file, why would you want
>> >> > to see .listFiles or other directory-specific methods?
>> >> >
>> >> > I'd go for File and Directory and not try to enforce that each
>> >> > actually refers to a physical regular file or directory.
>> >>
>> >> Also, except files and directories there are also nonexistent files
>> >> and symlinks (not counting devices and named sockets and pipes).
>> >>
>> >> If it is wrong to have listFiles on File object, then it is wrong to
>> >> have length() method on nonexistent file. So there must exists Path
>> >> base class for File and Directory. And also, why having ctime() method
>> >> on nonexistend object? So there must exists ExistingPath, subclass of
>> >> Path, and base class of File and Directory.
>> >>
>> >>
>> >> Look at typical code that deletes old files and directories.
>> >>
>> >> Old, java style:
>> >>
>> >> ===
>> >> def deleteOldFiles(dir: File) =
>> >>  for (file <- dir.list())
>> >>    if (file.isRegular() && file.ctime() < now - 3600)
>> >>      file.delete();
>> >> ===
>> >>
>> >> Proposed:
>> >>
>> >> ===
>> >> def deleteOldFiles(dir: Directory) =
>> >>  for (path <- dir.list())
>> >>    file match {
>> >>      case f: File =>
>> >>        if (file.ctime() < now - 3600)
>> >>          file.delete();
>> >>      case _ =>
>> >>    }
>> >> ===
>> >>
>> >> Old style is singificantly shorter and easier.
>> >>
>> >> I have to say one more thing. File is not a representation of "regular
>> >> file" or filesystem object. File represents a path on the filesystem.
>> >> Filesystem has "list children" operation. This operation is
>> >> represented by "list()" method of File object, that is interface to
>> >> the filesystem.
>> >>
>> >> S.
>> >
>> >
>> >
>> > --
>> > http://erikengbrecht.blogspot.com/
>> >
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On 07.09.2009, at 18:00, Miles Sabin wrote:

> On Mon, Sep 7, 2009 at 4:56 PM, Erik Engbrecht > wrote:
>> http://github.com/eengbrec/Scalax.IO/issues/#issue/2
>> There is no isSymlink function (there isn't on on java.io.File,
>> either), but
>
> No, but there is on java.nio.file.Path,
>
> http://download.java.net/jdk7/docs/api/java/nio/file/Path.html
>
> If you haven't already taken a very close look at this I suggest you
> do now.

I would also like to stress that point. A number of smart people have
put a significant amount of effort into NIO2, so we should definitely
be aware of what they've come up with. Apple's file-system APIs in
Snow Leopard are also worth a look maybe.

- Tiark

On Mon, Sep 7, 2009 at 4:56 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> http://github.com/eengbrec/Scalax.IO/issues/#issue/2
> There is no isSymlink function (there isn't on on java.io.File, either), but

No, but there is on java.nio.file.Path,

 http://download.java.net/jdk7/docs/api/java/nio/file/Path.html

If you haven't already taken a very close look at this I suggest you do now.

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

On Mon, Sep 7, 2009 at 5:43 PM, Erik Engbrecht wrote:
> I have taken a pretty close look at the API, but I haven't looked at the
> source.  But there's a couple issues:

Nevertheless, a great deal of work has gone into making the NIO2
filesystem API the highly capable, cross-platform interface and
implementation that Java has been missing for so long. Even if it were
desirable, that isn't something that the Scala community is going to
be able to replicate in a hurry, if at all.

Rather than reinventing the wheel it would make a great deal more
sense to treat the NIO2 API as raw material and build an idiomatic
Scala API on top of it.

The alternative is to build a pretty looking toy that gets dropped in
favour of the Java API as soon as anyone needs to do any serious work.
I don't think that would be a good strategy for building a standard
library for Scala.

Cheers,

Miles

AFAIU, Miles hasn't suggested to make scala.io dependent on JDK7. He
just said that it is possible to test if file is symlink.

S.

On Mon, Sep 7, 2009 at 20:43, Erik Engbrecht wrote:
> I have taken a pretty close look at the API, but I haven't looked at the
> source.  But there's a couple issues:
> 1. Introducing a dependency on Java 7 is a non-starter, so anything in Java
> 7 that we want needs to be ported into Scala rather than used directly.
> 2. I think much of the new IO stuff slated for Java 7 is at the wrong level
> of abstraction to be "user friendly," and I think Scala needs a more
> user-friendly IO package.  Low-level ops can always be accessed by calling
> directly into the Java libraries.
> I'm really aiming for something simple that can be relatively easily
> implemented through calls to Java 5 java.io classes and a little NIO when
> necessary.  I think it's an open question as to whether that would create
> the right level of abstraction for inclusion in the Scala standard library
> or not.  That also happens to be roughly the level that is currently in
> trunk.
> So in my opinion, this means that whatever goes in to 2.8 should be defined
> in terms of traits rather than concrete classes, so when Java 7 comes out it
> would be possible to write a more sophisticated API without breaking
> compatibility (assuming people code against the interfaces instead of the
> concrete implementations, but that can be forced by making the
> implementations private).
> -Erik
> On Mon, Sep 7, 2009 at 12:00 PM, Miles Sabin wrote:
>>
>> On Mon, Sep 7, 2009 at 4:56 PM, Erik Engbrecht
>> wrote:
>> > http://github.com/eengbrec/Scalax.IO/issues/#issue/2
>> > There is no isSymlink function (there isn't on on java.io.File, either),
>> > but
>>
>> No, but there is on java.nio.file.Path,
>>
>>  http://download.java.net/jdk7/docs/api/java/nio/file/Path.html
>>
>> If you haven't already taken a very close look at this I suggest you do
>> now.
>>
>> Cheers,
>>
>>
>> Miles
>>
>> --
>> Miles Sabin
>> tel: +44 (0)7813 944 528
>> skype:  milessabin
>> http://www.chuusai.com/
>> http://twitter.com/milessabin
>
>
>
> --
> http://erikengbrecht.blogspot.com/
>

On Mon, Sep 7, 2009 at 5:52 PM, Stepan Koltsov wrote:
> AFAIU, Miles hasn't suggested to make scala.io dependent on JDK7. He
> just said that it is possible to test if file is symlink.

I'm suggesting that we should aim to build any standard Scala
filesystem abstraction on the NIO2 implementation as soon as is
feasible.

I guess that's not quite the same as saying that scala.io should be
dependent on JDK7, but it's pretty close to it.

Frankly I think the chances of us replicating the work that's been
done on the NIO2 filesystem interface before JDK7 is released is close
to zero, so I don't really see that it'd do much harm to design the
Scala API with a view to releasing when JDK7 is available.

Cheers,

Miles

Hey Miles,

On Mon, 2009-09-07 at 17:56 +0100, Miles Sabin wrote:
> Nevertheless, a great deal of work has gone into making the NIO2
> filesystem API the highly capable, cross-platform interface and
> implementation that Java has been missing for so long.

I hope so, since the original plan was to include it in Java 5 (with a
proposed final draft by January 2004).

> Rather than reinventing the wheel it would make a great deal more
> sense to treat the NIO2 API as raw material and build an idiomatic
> Scala API on top of it.

Can you clarify what you mean here? Are you suggesting that Scala should
use NIO2 or that it should provide an API that is similar to NIO2 but
relying on JDK5 APIs? (some functionality would be lost in the latter
case, obviously).

> The alternative is to build a pretty looking toy that gets dropped in
> favour of the Java API as soon as anyone needs to do any serious work.
> I don't think that would be a good strategy for building a standard
> library for Scala.

I don't think that's the only alternative, but what you describe is
obviously something to be avoided. An interesting question is what
"serious work" means in this context.

Best,
Ismael

On Mon, Sep 7, 2009 at 6:43 PM, Erik Engbrecht wrote:
> 1. Introducing a dependency on Java 7 is a non-starter, so anything in Java
> 7 that we want needs to be ported into Scala rather than used directly.

Just out of curiosity: why?
I mean if people must stay at java 1.5, why can't the same people stay
at Scala 2.7?

BR,
John

On Mon, Sep 7, 2009 at 6:43 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> 1. Introducing a dependency on Java 7 is a non-starter, so anything in Java
> 7 that we want needs to be ported into Scala rather than used directly.

Just out of curiosity: why?
I mean if people must stay at java 1.5, why can't the same people stay
at Scala 2.7?

BR,
John

On Mon, Sep 7, 2009 at 6:14 PM, Ismael Juma wrote:
> Can you clarify what you mean here? Are you suggesting that Scala should
> use NIO2 or that it should provide an API that is similar to NIO2 but
> relying on JDK5 APIs? (some functionality would be lost in the latter
> case, obviously).

I'm saying that it should use NIO2. We want an API that's idiomatic
for Scala, obviously.

But there are some implementation constraints which are likely to show
through in a Scala API (at least, they will if it's done properly).

For instance NIO2 distinguishes files from directories via separately
queryable attributes rather than via Path subtypes because because,
eg., network filesystems make attribute queries extremely expensive.
Similarly it has an iterator-style API for directory contents because
an eager list-like API doesn't scale where directories can be very
large and/or slow to enumerate.

Getting this stuff right depends on being sensitive to the concrete
specifics as well as the abstract niceties, and NIO2 embodies years of
work on those specifics and we'd be foolish to ignore it.

That said, NIO2 _looks_ pretty ugly ... I think we can do a lot better in Scala.

Cheers,

Miles

How about if we build an abstraction that follows the design of NIO2, and isolate the functionality (via traits) that genuinely requires Java 7
For Linux users, there's also an NIO2 backport to Java 6 available: http://code.google.com/p/jsr203-backport/

On Mon, Sep 7, 2009 at 6:08 PM, John Nilsson <john@milsson.nu> wrote:

On Mon, Sep 7, 2009 at 6:43 PM, Erik Engbrecht<erik.engbrecht@gmail.com> wrote:
> 1. Introducing a dependency on Java 7 is a non-starter, so anything in Java
> 7 that we want needs to be ported into Scala rather than used directly.

Just out of curiosity: why?
I mean if people must stay at java 1.5, why can't the same people stay
at Scala 2.7?

BR,
John

On Mon, Sep 7, 2009 at 6:14 PM, Ismael Juma<mlists@juma.me.uk> wrote:
> Can you clarify what you mean here? Are you suggesting that Scala should
> use NIO2 or that it should provide an API that is similar to NIO2 but
> relying on JDK5 APIs? (some functionality would be lost in the latter
> case, obviously).

I'm saying that it should use NIO2. We want an API that's idiomatic
for Scala, obviously.

But there are some implementation constraints which are likely to show
through in a Scala API (at least, they will if it's done properly).

For instance NIO2 distinguishes files from directories via separately
queryable attributes rather than via Path subtypes because because,
eg., network filesystems make attribute queries extremely expensive.
Similarly it has an iterator-style API for directory contents because
an eager list-like API doesn't scale where directories can be very
large and/or slow to enumerate.

Getting this stuff right depends on being sensitive to the concrete
specifics as well as the abstract niceties, and NIO2 embodies years of
work on those specifics and we'd be foolish to ignore it.

That said, NIO2 _looks_ pretty ugly ... I think we can do a lot better in Scala.

Cheers,


Miles

--
Miles Sabin
tel: +44 (0)7813 944 528
skype:  milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

On 7 Sep 2009, at 18:45, Kevin Wright
wrote:

> Is this something we could handle with pluggable implementations, in
> a similar fashion to slf4j?
>
> It could be a good test case for scala modules :)

It would be good to define a "Scala-esque" API for representation of
the files/directories (e.g. one that could map over directory
contents) first, then have back-ends that can use whatever is
available. One could have an optional dependency on the Java-7 Scala
IO module and if it weren't available at runtime (say, not on the path
or not loadable) then it could fall back to other behaviours.

But I also think that designing it based on the N*IO libraries is a
bad idea - the IO APIs change every other release. We'll be getting
the new, new, new IO soon. It would be better to learn from orher's
mistakes (and the java.io is rich in learning opportunity) to get
something that feels right for Scala first.

Alec

On Mon, 2009-09-07 at 18:34 +0100, Miles Sabin wrote:
> I'm saying that it should use NIO2. We want an API that's idiomatic
> for Scala, obviously.

What is your suggestion for Scala 2.8.0 then? Do nothing?

> But there are some implementation constraints which are likely to show
> through in a Scala API (at least, they will if it's done properly).
>
> For instance NIO2 distinguishes files from directories via separately
> queryable attributes rather than via Path subtypes because because,
> eg., network filesystems make attribute queries extremely expensive.

Note that guarantees cannot be given even if querying is cheap since the
filesystem is mutable. Personally, I'd like for a way to specify intent
in my method signatures instead of scaladoc. As far as I can see, NIO2
doesn't allow that, so I believe there's room for improvement there.

> Similarly it has an iterator-style API for directory contents because
> an eager list-like API doesn't scale where directories can be very
> large and/or slow to enumerate.

Of course. What is hard to believe is that there's no such API in any of
the currently released JDKs. I hope we're not going in the same
direction with Scala by delaying things until we can get every detail
right.

> Getting this stuff right depends on being sensitive to the concrete
> specifics as well as the abstract niceties, and NIO2 embodies years of
> work on those specifics and we'd be foolish to ignore it.

No-one is ignoring it. Both Paul and Erik have looked at NIO2 (and
continue doing so).

Personally, I believe that better IO for scala 2.8.0 is important and
waiting for JDK7 (that doesn't even have a release date) before
improving it is not the best way forward. Also, even once JDK7 is out,
it's unclear how long it will take before Scala can depend on it.

One option is to mark the new IO classes as experimental to warn users
that changes may be made in the future.

Best,
Ismael