This page is no longer maintained — Please continue to the home page at www.scala-lang.org

indexing scala irc channel

15 replies
lucastorri
Joined: 2011-02-15,
User offline. Last seen 42 years 45 weeks ago.
Hello everybody,
for a little more than a month now, I've been logging all conversations on the Scala IRC channel. All those logs are being indexed by Lucene and I made them searchable through a Play 2.0 application. Communication between the play app and the one running Lucene are done through Akka, and through WebSockets between browser and server (so it might not work properly in some browsers). It can be accessed on http://ircsearch.latestbuild.net/
It's just a quick experiment I've been doing on my free time to try those frameworks/technologies. I hope you like.
Here are a few searches you can try:
The code is avaliable on my github (https://github.com/lucastorri) and soon I plan to describe on my blog details of how it all works together (http://latestbuild.net/).
That's just an initial version. I'll be doing some improvements on it as my free time allows to do so (like using akka 2.0 on the lucene app, upgrade to the latest play version, have it use some communication channel other than websockets, etc).
Cheers
--
Lucas B. Torri
alois.cochard
Joined: 2010-03-19,
User offline. Last seen 34 weeks 2 hours ago.
Re: indexing scala irc channel

Hello Lucas,

I'm sorry to say but I don't like having my conversation shared
publicly on the web.

Even it's public chat, I don't except that what I write on IRC will
suddenly be published on the web... don't know what others IRC users
think of that, maybe I'm the only one to think of it that way...

Sometimes I share personal point of view about technology, not
something I say publicly on twitter or on my blog.

Alois

On Feb 15, 3:17 pm, Lucas Torri wrote:
> Hello everybody,
>
> for a little more than a month now, I've been logging all conversations on
> the Scala IRC channel. All those logs are being indexed by Lucene and I
> made them searchable through a Play 2.0 application. Communication between
> the play app and the one running Lucene are done through Akka, and through
> WebSockets between browser and server (so it might not work properly in
> some browsers). It can be accessed onhttp://ircsearch.latestbuild.net/
>
> It's just a quick experiment I've been doing on my free time to try those
> frameworks/technologies. I hope you like.
>
> Here are a few searches you can try:
>
>    -http://ircsearch.latestbuild.net/#scalazmemoization
>    -http://ircsearch.latestbuild.net/#scalajava clojure
>
> The code is avaliable on my github (https://github.com/lucastorri) and soon
> I plan to describe on my blog details of how it all works together (http://latestbuild.net/).
>
> That's just an initial version. I'll be doing some improvements on it as my
> free time allows to do so (like using akka 2.0 on the lucene app, upgrade
> to the latest play version, have it use some communication channel other
> than websockets, etc).
>
> Cheers
>
> --
> Lucas B. Torri

lucastorri
Joined: 2011-02-15,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: indexing scala irc channel
I was expecting reactions to it. It's good to remember that the bot supports the "[mute]" tag. Whenever that string appears on a message, it won't be logged.
Anyway, if most people dislike it I have no choice other than shut it down.

On Wed, Feb 15, 2012 at 12:41 PM, Alois Cochard <alois.cochard@gmail.com> wrote:
Hello Lucas,

I'm sorry to say but I don't like having my conversation shared
publicly on the web.

Even it's public chat, I don't except that what I write on IRC will
suddenly be published on the web... don't know what others IRC users
think of that, maybe I'm the only one to think of it that way...

Sometimes I share personal point of view about technology, not
something I say publicly on twitter or on my blog.

Alois

On Feb 15, 3:17 pm, Lucas Torri <lucasto...@gmail.com> wrote:
> Hello everybody,
>
> for a little more than a month now, I've been logging all conversations on
> the Scala IRC channel. All those logs are being indexed by Lucene and I
> made them searchable through a Play 2.0 application. Communication between
> the play app and the one running Lucene are done through Akka, and through
> WebSockets between browser and server (so it might not work properly in
> some browsers). It can be accessed onhttp://ircsearch.latestbuild.net/
>
> It's just a quick experiment I've been doing on my free time to try those
> frameworks/technologies. I hope you like.
>
> Here are a few searches you can try:
>
>    -http://ircsearch.latestbuild.net/#scalazmemoization
>    -http://ircsearch.latestbuild.net/#scalajava clojure
>
> The code is avaliable on my github (https://github.com/lucastorri) and soon
> I plan to describe on my blog details of how it all works together (http://latestbuild.net/).
>
> That's just an initial version. I'll be doing some improvements on it as my
> free time allows to do so (like using akka 2.0 on the lucene app, upgrade
> to the latest play version, have it use some communication channel other
> than websockets, etc).
>
> Cheers
>
> --
> Lucas B. Torri



--
Lucas B. Torri
alois.cochard
Joined: 2010-03-19,
User offline. Last seen 34 weeks 2 hours ago.
Re: indexing scala irc channel

Well,

It should be at least explained in the channel topic according to
freenode guidelines: http://freenode.net/channel_guidelines.shtml

As a side note I would have preferred that you ask for that *BEFORE*
publishing more than one moth of log online.

Here is the relevant part:

===============================
If you're considering publishing channel logs, think it through. The
freenode network is an interactive environment. Even on public
channels, most users don't weigh their comments with the idea that
they'll be enshrined in perpetuity. For that reason, few participants
publish logs.

If you're publishing logs on an ongoing basis, your channel topic
should reflect that fact. Be sure to provide a way for users to make
comments without logging, and get permission from the channel owners
before you start. If you're thinking of "anonymizing" your logs
(removing information that identifies the specific users), be aware
that it's difficult to do it well—replies and general context often
provide identifying information which is hard to filter.

If you just want to publish a single conversation, be careful to get
permission from each participant. Provide as much context as you can.
Avoid the temptation to publish or distribute logs without permission
in order to portray someone in a bad light. The reputation you save
will most likely be your own.
===============================

Cheers,

On Feb 15, 3:46 pm, Lucas Torri wrote:
> I was expecting reactions to it. It's good to remember that the bot
> supports the "[mute]" tag. Whenever that string appears on a message, it
> won't be logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.
>
> On Wed, Feb 15, 2012 at 12:41 PM, Alois Cochard wrote:
>
>
>
>
>
>
>
>
>
> > Hello Lucas,
>
> > I'm sorry to say but I don't like having my conversation shared
> > publicly on the web.
>
> > Even it's public chat, I don't except that what I write on IRC will
> > suddenly be published on the web... don't know what others IRC users
> > think of that, maybe I'm the only one to think of it that way...
>
> > Sometimes I share personal point of view about technology, not
> > something I say publicly on twitter or on my blog.
>
> > Alois
>
> > On Feb 15, 3:17 pm, Lucas Torri wrote:
> > > Hello everybody,
>
> > > for a little more than a month now, I've been logging all conversations
> > on
> > > the Scala IRC channel. All those logs are being indexed by Lucene and I
> > > made them searchable through a Play 2.0 application. Communication
> > between
> > > the play app and the one running Lucene are done through Akka, and
> > through
> > > WebSockets between browser and server (so it might not work properly in
> > > some browsers). It can be accessed onhttp://ircsearch.latestbuild.net/
>
> > > It's just a quick experiment I've been doing on my free time to try those
> > > frameworks/technologies. I hope you like.
>
> > > Here are a few searches you can try:
>
> > >    -http://ircsearch.latestbuild.net/#scalazmemoization
> > >    -http://ircsearch.latestbuild.net/#scalajavaclojure
>
> > > The code is avaliable on my github (https://github.com/lucastorri) and
> > soon
> > > I plan to describe on my blog details of how it all works together (
> >http://latestbuild.net/).
>
> > > That's just an initial version. I'll be doing some improvements on it as
> > my
> > > free time allows to do so (like using akka 2.0 on the lucene app, upgrade
> > > to the latest play version, have it use some communication channel other
> > > than websockets, etc).
>
> > > Cheers
>
> > > --
> > > Lucas B. Torri
>
> --
> Lucas B. Torri

Jason Zaugg
Joined: 2009-05-18,
User offline. Last seen 38 weeks 5 days ago.
Re: Re: indexing scala irc channel

On Wed, Feb 15, 2012 at 3:41 PM, Alois Cochard wrote:
> Hello Lucas,
>
> I'm sorry to say but I don't like having my conversation shared
> publicly on the web.

+1

-jason

lucastorri
Joined: 2011-02-15,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: indexing scala irc channel
I brought it down (although the front end is still accessible). The logs are not accessible nor searchable anymore.
Thank you for the advices.

On Wed, Feb 15, 2012 at 1:10 PM, Alois Cochard <alois.cochard@gmail.com> wrote:
Well,

It should be at least explained in the channel topic according to
freenode guidelines: http://freenode.net/channel_guidelines.shtml

As a side note I would have preferred that you ask for that *BEFORE*
publishing more than one moth of log online.

Here is the relevant part:

===============================
If you're considering publishing channel logs, think it through. The
freenode network is an interactive environment. Even on public
channels, most users don't weigh their comments with the idea that
they'll be enshrined in perpetuity. For that reason, few participants
publish logs.

If you're publishing logs on an ongoing basis, your channel topic
should reflect that fact. Be sure to provide a way for users to make
comments without logging, and get permission from the channel owners
before you start. If you're thinking of "anonymizing" your logs
(removing information that identifies the specific users), be aware
that it's difficult to do it well—replies and general context often
provide identifying information which is hard to filter.

If you just want to publish a single conversation, be careful to get
permission from each participant. Provide as much context as you can.
Avoid the temptation to publish or distribute logs without permission
in order to portray someone in a bad light. The reputation you save
will most likely be your own.
===============================

Cheers,

On Feb 15, 3:46 pm, Lucas Torri <lucasto...@gmail.com> wrote:
> I was expecting reactions to it. It's good to remember that the bot
> supports the "[mute]" tag. Whenever that string appears on a message, it
> won't be logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.
>
> On Wed, Feb 15, 2012 at 12:41 PM, Alois Cochard <alois.coch...@gmail.com>wrote:
>
>
>
>
>
>
>
>
>
> > Hello Lucas,
>
> > I'm sorry to say but I don't like having my conversation shared
> > publicly on the web.
>
> > Even it's public chat, I don't except that what I write on IRC will
> > suddenly be published on the web... don't know what others IRC users
> > think of that, maybe I'm the only one to think of it that way...
>
> > Sometimes I share personal point of view about technology, not
> > something I say publicly on twitter or on my blog.
>
> > Alois
>
> > On Feb 15, 3:17 pm, Lucas Torri <lucasto...@gmail.com> wrote:
> > > Hello everybody,
>
> > > for a little more than a month now, I've been logging all conversations
> > on
> > > the Scala IRC channel. All those logs are being indexed by Lucene and I
> > > made them searchable through a Play 2.0 application. Communication
> > between
> > > the play app and the one running Lucene are done through Akka, and
> > through
> > > WebSockets between browser and server (so it might not work properly in
> > > some browsers). It can be accessed onhttp://ircsearch.latestbuild.net/
>
> > > It's just a quick experiment I've been doing on my free time to try those
> > > frameworks/technologies. I hope you like.
>
> > > Here are a few searches you can try:
>
> > >    -http://ircsearch.latestbuild.net/#scalazmemoization
> > >    -http://ircsearch.latestbuild.net/#scalajavaclojure
>
> > > The code is avaliable on my github (https://github.com/lucastorri) and
> > soon
> > > I plan to describe on my blog details of how it all works together (
> >http://latestbuild.net/).
>
> > > That's just an initial version. I'll be doing some improvements on it as
> > my
> > > free time allows to do so (like using akka 2.0 on the lucene app, upgrade
> > > to the latest play version, have it use some communication channel other
> > > than websockets, etc).
>
> > > Cheers
>
> > > --
> > > Lucas B. Torri
>
> --
> Lucas B. Torri



--
Lucas B. Torri
milessabin
Joined: 2008-08-11,
User offline. Last seen 33 weeks 3 days ago.
Re: Re: indexing scala irc channel

On Wed, Feb 15, 2012 at 2:46 PM, Lucas Torri wrote:
> I was expecting reactions to it. It's good to remember that the bot supports
> the "[mute]" tag. Whenever that string appears on a message, it won't be
> logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.

I'm very much in favour ... I think having publicly visible and
searchable logs dramatically increases the value of the channel.

Cheers,

Miles

Kevin Wright 2
Joined: 2010-05-30,
User offline. Last seen 26 weeks 4 days ago.
Re: Re: indexing scala irc channel
Total agreement...
I've seen lots of useful information come out of IRC, it's a shame to think that much of that is now lost.

If you want to be anonymous on IRC, use an alias and don't tell anyone who you are.  It's not Google+

On 15 February 2012 17:00, Miles Sabin <miles@milessabin.com> wrote:
On Wed, Feb 15, 2012 at 2:46 PM, Lucas Torri <lucastorri@gmail.com> wrote:
> I was expecting reactions to it. It's good to remember that the bot supports
> the "[mute]" tag. Whenever that string appears on a message, it won't be
> logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.

I'm very much in favour ... I think having publicly visible and
searchable logs dramatically increases the value of the channel.

Cheers,


Miles


Christian
Joined: 2012-02-15,
User offline. Last seen 42 years 45 weeks ago.
Re: Re: indexing scala irc channel
If the data shouldn't be public, how did Lucas accessed it ?

2012/2/15 Kevin Wright <kev.lee.wright@gmail.com>
Total agreement...
I've seen lots of useful information come out of IRC, it's a shame to think that much of that is now lost.

If you want to be anonymous on IRC, use an alias and don't tell anyone who you are.  It's not Google+

On 15 February 2012 17:00, Miles Sabin <miles@milessabin.com> wrote:
On Wed, Feb 15, 2012 at 2:46 PM, Lucas Torri <lucastorri@gmail.com> wrote:
> I was expecting reactions to it. It's good to remember that the bot supports
> the "[mute]" tag. Whenever that string appears on a message, it won't be
> logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.

I'm very much in favour ... I think having publicly visible and
searchable logs dramatically increases the value of the channel.

Cheers,


Miles



Alex Cruise
Joined: 2008-12-17,
User offline. Last seen 2 years 26 weeks ago.
Re: indexing scala irc channel
On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri <lucastorri@gmail.com> wrote:
for a little more than a month now, I've been logging all conversations on the Scala IRC channel.

I'm fine with publicly logged IRC channels, as long as it's a deliberate and explicitly declared policy decision by the channel owner/operator(s) rather than an autonomous act by a channel participant.  
To my knowledge, no such decision has ever previously been made or announced with respect to #scala, so I think it's best that this data set be deleted.  
If the owner/operator(s) decide that the channel should be logged in future, or someone decides to start another, logged channel ( maybe ##scala?) your project seems like it would be a good fit.
I hope this experience doesn't demotivate you.  Keep hakking! :)
-0xe1a
lucastorri
Joined: 2011-02-15,
User offline. Last seen 42 years 45 weeks ago.
Re: indexing scala irc channel
who is responsible for the channel?

On Wed, Feb 15, 2012 at 4:54 PM, Alex Cruise <alex@cluonflux.com> wrote:
On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri <lucastorri@gmail.com> wrote:
for a little more than a month now, I've been logging all conversations on the Scala IRC channel.

I'm fine with publicly logged IRC channels, as long as it's a deliberate and explicitly declared policy decision by the channel owner/operator(s) rather than an autonomous act by a channel participant.  
To my knowledge, no such decision has ever previously been made or announced with respect to #scala, so I think it's best that this data set be deleted.  
If the owner/operator(s) decide that the channel should be logged in future, or someone decides to start another, logged channel ( maybe ##scala?) your project seems like it would be a good fit.
I hope this experience doesn't demotivate you.  Keep hakking! :)
-0xe1a



--
Lucas B. Torri
Peter Salanki
Joined: 2010-01-27,
User offline. Last seen 42 years 45 weeks ago.
Re: indexing scala irc channel
On Feb 15, 2012, at 12:09 PM, Kevin Wright wrote:
Total agreement...
I've seen lots of useful information come out of IRC, it's a shame to think that much of that is now lost.
Exactly, +1

If you want to be anonymous on IRC, use an alias and don't tell anyone who you are.  It's not Google+


On 15 February 2012 17:00, Miles Sabin <miles@milessabin.com> wrote:
On Wed, Feb 15, 2012 at 2:46 PM, Lucas Torri <lucastorri@gmail.com> wrote:
> I was expecting reactions to it. It's good to remember that the bot supports
> the "[mute]" tag. Whenever that string appears on a message, it won't be
> logged.
>
> Anyway, if most people dislike it I have no choice other than shut it down.

I'm very much in favour ... I think having publicly visible and
searchable logs dramatically increases the value of the channel.

Cheers,


Miles



alois.cochard
Joined: 2010-03-19,
User offline. Last seen 34 weeks 2 hours ago.
Re: indexing scala irc channel

On 15 February 2012 19:54, Alex Cruise <alex@cluonflux.com> wrote:
On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri <lucastorri@gmail.com> wrote:
for a little more than a month now, I've been logging all conversations on the Scala IRC channel.

I'm fine with publicly logged IRC channels, as long as it's a deliberate and explicitly declared policy decision by the channel owner/operator(s) rather than an autonomous act by a channel participant.  
To my knowledge, no such decision has ever previously been made or announced with respect to #scala, so I think it's best that this data set be deleted.  

Totally agree, I'm open to having IRC logged publicly but I would prefer to know that when I'm in the channel (in channel topic), and not retroactively without knowing, and without a decision of channel OP.

That's not a matter of defining if this kind of information are considered public or not, it's just obvious netiquette to me.

--
Alois Cochardhttp://aloiscochard.blogspot.com http://twitter.com/aloiscochard
Matthew Pocock 3
Joined: 2010-07-30,
User offline. Last seen 42 years 45 weeks ago.
Re: indexing scala irc channel
I would prefer the IRC channel to be logged and publicly indexed and/or exposed to google. There's loads of useful stuff on there which is often more accessible in that format than in blogs or books. However, I understand that people sometimes use it for messages that they may not expect to be kept for posterity.
Matthew

On 15 February 2012 21:03, Alois Cochard <alois.cochard@gmail.com> wrote:

On 15 February 2012 19:54, Alex Cruise <alex@cluonflux.com> wrote:
On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri <lucastorri@gmail.com> wrote:
for a little more than a month now, I've been logging all conversations on the Scala IRC channel.

I'm fine with publicly logged IRC channels, as long as it's a deliberate and explicitly declared policy decision by the channel owner/operator(s) rather than an autonomous act by a channel participant.  
To my knowledge, no such decision has ever previously been made or announced with respect to #scala, so I think it's best that this data set be deleted.  

Totally agree, I'm open to having IRC logged publicly but I would prefer to know that when I'm in the channel (in channel topic), and not retroactively without knowing, and without a decision of channel OP.

That's not a matter of defining if this kind of information are considered public or not, it's just obvious netiquette to me.

--
Alois Cochard http://aloiscochard.blogspot.com http://twitter.com/aloiscochard



--
Dr Matthew PocockIntegrative Bioinformatics Group, School of Computing Science, Newcastle Universitymailto: turingatemyhamster@gmail.com gchat: turingatemyhamster@gmail.commsn: matthew_pocock@yahoo.co.uk irc.freenode.net: drdozerskype: matthew.pococktel: (0191) 2566550mob: +447535664143
Viktor Klang
Joined: 2008-12-17,
User offline. Last seen 1 year 27 weeks ago.
Re: indexing scala irc channel

On Mon, Feb 20, 2012 at 1:01 PM, Matthew Pocock
wrote:
> I would prefer the IRC channel to be logged and publicly indexed and/or
> exposed to google. There's loads of useful stuff on there which is often
> more accessible in that format than in blogs or books. However, I understand
> that people sometimes use it for messages that they may not expect to be
> kept for posterity.

Add some sort of escape for off-the-record messages?

Cheers,

>
> Matthew
>
>
> On 15 February 2012 21:03, Alois Cochard wrote:
>>
>>
>> On 15 February 2012 19:54, Alex Cruise wrote:
>>>
>>> On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri
>>> wrote:
>>>>
>>>> for a little more than a month now, I've been logging all conversations
>>>> on the Scala IRC channel.
>>>
>>>
>>> I'm fine with publicly logged IRC channels, as long as it's a deliberate
>>> and explicitly declared policy decision by the channel owner/operator(s)
>>> rather than an autonomous act by a channel participant.
>>>
>>> To my knowledge, no such decision has ever previously been made or
>>> announced with respect to #scala, so I think it's best that this data set be
>>> deleted.
>>>
>>
>> Totally agree, I'm open to having IRC logged publicly but I would prefer
>> to know that when I'm in the channel (in channel topic), and not
>> retroactively without knowing, and without a decision of channel OP.
>>
>> That's not a matter of defining if this kind of information are considered
>> public or not, it's just obvious netiquette to me.
>>
>> --
>> Alois Cochard
>> http://aloiscochard.blogspot.com
>> http://twitter.com/aloiscochard
>>
>
>
>
> --
> Dr Matthew Pocock
> Integrative Bioinformatics Group, School of Computing Science, Newcastle
> University
> mailto: turingatemyhamster@gmail.com
> gchat: turingatemyhamster@gmail.com
> msn: matthew_pocock@yahoo.co.uk
> irc.freenode.net: drdozer
> skype: matthew.pocock
> tel: (0191) 2566550
> mob: +447535664143
>

Joshua.Suereth
Joined: 2008-09-02,
User offline. Last seen 32 weeks 5 days ago.
Re: indexing scala irc channel

If you get the moderator permission and it's in the welcome message that things are recorded (with a mute option), I don't see much harm.  It's the surprise of unexpected publicity that people were upset about.

On Feb 20, 2012 8:27 AM, "√iktor Ҡlang" <viktor.klang@gmail.com> wrote:
On Mon, Feb 20, 2012 at 1:01 PM, Matthew Pocock
<turingatemyhamster@gmail.com> wrote:
> I would prefer the IRC channel to be logged and publicly indexed and/or
> exposed to google. There's loads of useful stuff on there which is often
> more accessible in that format than in blogs or books. However, I understand
> that people sometimes use it for messages that they may not expect to be
> kept for posterity.

Add some sort of escape for off-the-record messages?

Cheers,


>
> Matthew
>
>
> On 15 February 2012 21:03, Alois Cochard <alois.cochard@gmail.com> wrote:
>>
>>
>> On 15 February 2012 19:54, Alex Cruise <alex@cluonflux.com> wrote:
>>>
>>> On Wed, Feb 15, 2012 at 6:17 AM, Lucas Torri <lucastorri@gmail.com>
>>> wrote:
>>>>
>>>> for a little more than a month now, I've been logging all conversations
>>>> on the Scala IRC channel.
>>>
>>>
>>> I'm fine with publicly logged IRC channels, as long as it's a deliberate
>>> and explicitly declared policy decision by the channel owner/operator(s)
>>> rather than an autonomous act by a channel participant.
>>>
>>> To my knowledge, no such decision has ever previously been made or
>>> announced with respect to #scala, so I think it's best that this data set be
>>> deleted.
>>>
>>
>> Totally agree, I'm open to having IRC logged publicly but I would prefer
>> to know that when I'm in the channel (in channel topic), and not
>> retroactively without knowing, and without a decision of channel OP.
>>
>> That's not a matter of defining if this kind of information are considered
>> public or not, it's just obvious netiquette to me.
>>
>> --
>> Alois Cochard
>> http://aloiscochard.blogspot.com
>> http://twitter.com/aloiscochard
>>
>
>
>
> --
> Dr Matthew Pocock
> Integrative Bioinformatics Group, School of Computing Science, Newcastle
> University
> mailto: turingatemyhamster@gmail.com
> gchat: turingatemyhamster@gmail.com
> msn: matthew_pocock@yahoo.co.uk
> irc.freenode.net: drdozer
> skype: matthew.pocock
> tel: (0191) 2566550
> mob: +447535664143
>



--
Viktor Klang

Akka Tech Lead
Typesafe - The software stack for applications that scale

Twitter: @viktorklang

Copyright © 2012 École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland