lighttpd forum lighty > Should we build a .htaccess-like module?

[poco] so, what's the point with .htaccess-like support ? is anyone 
working on it ?
[poco] if not i would be glad with some people i know to do it, maybe as 
a module
[poco] because i strongly need it, but if someone is already working on 
it, just tell me
I also read that it may be interesting to have it inside the mysql_vhost 
module?

If no one is interested, I will work on it next week. We (webhosting 
services) really need dynamic configuration per user, without having to 
reload the whole server.

Thanks for this super web server :) I love it.

--
Neil Dökkalfar

Neil Dökkalfar wrote:
> [poco] so, what's the point with .htaccess-like support ? is anyone 
> working on it ?
> [poco] if not i would be glad with some people i know to do it, maybe as 
> a module
> [poco] because i strongly need it, but if someone is already working on 
> it, just tell me
> I also read that it may be interesting to have it inside the mysql_vhost 
> module?
> 
> If no one is interested, I will work on it next week. We (webhosting 
> services) really need dynamic configuration per user, without having to 
> reload the whole server.
> 
> Thanks for this super web server :) I love it.
> 
> --
> Neil Dökkalfar

yeaha, thats a great idea. maybe you should ask jan if he knows someone 
who is already working on such a module.
(but i think theres isnt anyone).

mfg
Ingo

Agree, this will be great feature.

P.S. Lighty has the option to use fam cache. If fam is on, I think that 
files like .htaccess must be checked using fam too, because of 
performance.

This is a main problem why people can't switch to lighty because they've 
alot of vhosts. And one update will mean a restart of the whole lighttpd 
server. It'll be a great feature and I'd support you 100% on it. For the 
last few months I've been with lighty I haven't seen anyone say anything 
about making such a mod.

So please make it =) ;)

Neil Dökkalfar wrote:
> [poco] so, what's the point with .htaccess-like support ? is anyone 
> working on it ?
> [poco] if not i would be glad with some people i know to do it, maybe as 
> a module
> [poco] because i strongly need it, but if someone is already working on 
> it, just tell me
> I also read that it may be interesting to have it inside the mysql_vhost 
> module?

I say no, please do not create such a module. One of the reasons that 
Lighttpd is so much faster and simpler than Apache is the lack of the 
"kitchen sink" concept that is Apache: checking for .htaccess files, 
especially recursively up the directory tree, is very slow. We must do 
everything we can to keep Lighttpd highly performant.

Figure out another way to address the problem. .htaccess isn't the 
answer.

- t

thomas lackner wrote:
> Neil Dökkalfar wrote:
>> [poco] so, what's the point with .htaccess-like support ? is anyone 
>> working on it ?
>> [poco] if not i would be glad with some people i know to do it, maybe as 
>> a module
>> [poco] because i strongly need it, but if someone is already working on 
>> it, just tell me
>> I also read that it may be interesting to have it inside the mysql_vhost 
>> module?
> 
> I say no, please do not create such a module. One of the reasons that 
> Lighttpd is so much faster and simpler than Apache is the lack of the 
> "kitchen sink" concept that is Apache: checking for .htaccess files, 
> especially recursively up the directory tree, is very slow. We must do 
> everything we can to keep Lighttpd highly performant.
> 
> Figure out another way to address the problem. .htaccess isn't the 
> answer.
> 
> - t

What other way could there be to address this problem?

Reloading the .conf every time a different directory is accessed?

Wouldn't something like that impact performance as well?

> If no one is interested, I will work on it next week. We (webhosting 
no, go for it. but u'd better think and discuss about it before coding.
irc://irc.freenode.org/lighttpd
> services) really need dynamic configuration per user, without having to 
> reload the whole server.
> 
> Thanks for this super web server :) I love it.

Yes, please! Make mod for .htaccess with mod_rewrite (like in apache) 
support! I need it too for my local webserver. It will be very succeful.

thomas lackner wrote:
> Neil Dökkalfar wrote:
>> [poco] so, what's the point with .htaccess-like support ? is anyone 
>> working on it ?
>> [poco] if not i would be glad with some people i know to do it, maybe as 
>> a module
>> [poco] because i strongly need it, but if someone is already working on 
>> it, just tell me
>> I also read that it may be interesting to have it inside the mysql_vhost 
>> module?
> 
> I say no, please do not create such a module. One of the reasons that 
> Lighttpd is so much faster and simpler than Apache is the lack of the 
> "kitchen sink" concept that is Apache: checking for .htaccess files, 
> especially recursively up the directory tree, is very slow. We must do 
> everything we can to keep Lighttpd highly performant.
> 
> Figure out another way to address the problem. .htaccess isn't the 
> answer.
> 
> - t

thats why its a module. if you dont need the support, dont load it. it
isnt gonna affect the performance if you don't use it.

we would not even NEED htaccess if we had a better vhost system.
let me explain:
what is the advantage of htaccess?
many ppl use it, because it offers ADHOC and instantly changes to a 
directory or site-wide configuration WITHOUT (!!!) any need to 
reload/restart the server and its configuration.

what i want to say is. we dont need htaccess. we need a solution which 
offers adhoc and instant changes to the configuration WITHOUT the need 
to restart lighty.

we have some great "beginning" modules available but due a matter of 
time or lust of the originate developer e.g. the mysql vhost module is 
still after over a year rudimentary.

nearly a year ago i already stated a suggestion that the mysql table 
should be extended by more features. the most "simple" way would be to 
add a text field in which an admin could write anything which is usually 
written in a $HTTP["host"] directive (e.g. rewrites redirects urldenies 
authentication ...). it would not be a real hard performance lost since 
mysql would cache resultsets if the data in the table or row hasnt 
changed.

but what would that offer to us? well there would be no need for 
htaccess anymore. EVEN BETTER - we would have a REAL vhost solution 
which is also a lot better to administrate. if an admin would change 
something it would be on air instantly! also its more secure. admins 
could offer webinterfaces to users so they can edit some content 
themself. on the otherhand admins could refuse any sporadic incidents 
offering security holes (e.g. if a user would change the htaccess file 
of another webuser because the admin has set the posix rights wrong.

i hope i stated my suggestion clear enough.

just imagine you could change ANY vhost related directives in realtime 
by changing a field in the database... :D

Simon

I see two drawbacks to use put it in the mysql_vhost module: the need of 
using mysql; and the need of an external interface. Otherwise, with a 
mysql database and an interface it can be nice too. (even if i don't 
like mysql :p)

An external interface is needed in order to allow clients insert their 
personal config in the database (and checking it, but this is also true 
for the other way).

I think the file based htaccess wouldn't require a lighty restart and, 
as proposed by Vyacheslav Chernousov it can also be implemented using 
fam.

--
Neil "poco" Dökkalfar <neil_at_next-touch_dot_com>

Neil Dökkalfar wrote:
> I see two drawbacks to use put it in the mysql_vhost module: the need of 
> using mysql; and the need of an external interface. Otherwise, with a 
> mysql database and an interface it can be nice too. (even if i don't 
> like mysql :p)
interfaces exist in many ways phpmyadmin webmin etc.
also it is not very complicated to code a 5min project to allow 
customers to edit their vhost settings - its even more secure. ;)

> An external interface is needed in order to allow clients insert their 
> personal config in the database (and checking it, but this is also true 
> for the other way).
pretty easy as mentioned above. htaccess has many disadavantages as 
performance and fam wont solve many issues where mysql and the existing 
vhost module already offering a working solution.

> I think the file based htaccess wouldn't require a lighty restart and, 
> as proposed by Vyacheslav Chernousov it can also be implemented using 
> fam.
i prefer a advanced solution and not stepping on the same place as only 
adopting apache's htaccess system. htaccess was nice for the 90th. now 
we are in the 21st century. ;)

Simon

Hi,

that's a very good idea. I was searching for such a module but couldn't 
find anything. It can be very useful because there is no need to 
"translate" the .htaccess files. Having made a change in the 
configuration, the server has to be restarted. To avoid that problem 
such module would help. Also I don't think there will be *VERY high* 
performance losses: If they are higher than expected, you could think 
about caching the .htaccess files. That reduces the number of hard disk 
accesses and the module just would have to load the content from the 
cache.
The reason why I'm looking for a .htaccess module is, that there are 
many scripts using .htaccess. I was developing my scripts with 
.htaccess, too. Therefore it's much work to "translate" every file 
manually. Because our server had high performance problems (server was 
not anymore reachable), we had to switch to LightTPD. I'm very confident 
with it, but the only thing I miss, is the .htaccess support.
What's the progress of the development? As I see the last reply has been 
made a long time ago. If the development of the project was stopped, a 
conformation application would be also a good alternative to minimize 
the work. That conformation application would translate the .htaccess 
files into the LightTPD format.

Thank you in advance for your answers.

Best regards
Tim

I'm interested in that too.
Thats the only real thing keeping me from switching to lighttpd.

Any progress on that?

Oh and I wouldn't mind it its build into the mysql-vhost module even 
though I'm not using that yet :).


Thanks,

thomas

Hi I think that this is a good idea except instead of opening up the
.htaccess file in the directory and scanning it, and recursively doing
the same for all of the folders above it, which would be very slow,
instead it should just read them at startup and store them in memory.
But with a file that when you execute it reloads them. This way you can
change the .htaccess files like you do in Apache, and then just run that
file, and it reloads the .htaccess files into memory, instead of doing
it on every request. This way there is no performance hit. And if you
use something like PHP to dynamically modify them automatically under
certain circumstances, then just have PHP run that file when it's done.

So ya basically be able to reload the config files on the fly at any 
time without having to restart the server, and so it doesn't have to 
keep checking the files over and over looking for changes.

I'd rather like to see lighttpd supporting reloading config files 
on-the-fly (probably using a signal).
This way people wanting such .htaccess-like files could easily implement 
that externally (so lighty isn't cluttered up with such code), generate 
a global config file from it and make lighttpd reload its config files.
I guess this is rather hard to do though...

I am for the .htaccess module too if it would use the same syntax as 
Apache :)

Well, what about doing something like this.  Give every user a "config" 
file in their webroot (call it .lighttpd).  They can control what ever 
they want in there, but when lighttpd loads it, it loads it in the 
$HTTP["HOST"] conditional (as to not damage others sites).  Now, the 
server stats the file on every nth request (or do a random number 
percent, so it winds up not stating the file every request).  If the 
stat changes, then re-load the file.  If the file fails (syntax), revert 
to the previous settings... So that any changes have a near (for all 
practical purposes) instentaneous effect...

Anthony Ferrara wrote:
> Well, what about doing something like this.  Give every user a "config" 
> file in their webroot (call it .lighttpd).  They can control what ever 
> they want in there, but when lighttpd loads it, it loads it in the 
> $HTTP["HOST"] conditional (as to not damage others sites).  Now, the 
> server stats the file on every nth request (or do a random number 
> percent, so it winds up not stating the file every request).  If the 
> stat changes, then re-load the file.  If the file fails (syntax), revert 
> to the previous settings... So that any changes have a near (for all 
> practical purposes) instentaneous effect...

This would only take the changes into effect every X requests.
And it would be wasting time checking the file when it doesn't need to.

Azu    wrote:
> This would only take the changes into effect every X requests.
> And it would be wasting time checking the file when it doesn't need to.

What about using fam/gamin for monitoring these files?
That way lighttpd wouldn't need to check them over and over again.

But on the other side I wouldn't have a problem telling lighttpd to 
reload this and that file if there would be an API to do so without 
restarting lighttpd alltogether ;).
IMO this would be the best way: some API to dynamically change the 
configuration - but I guess thats nearly impossible.
That way someone could build a separate application monitoring/searching 
for these files and sending their changes to lighttpd.

Any way to get some dynamic configuration working would be fine with me 
but I guess as long as its only users talking about it and no one 
willing to implement it, its rather useless :(.

Neil Dökkalfar wrote:
> If no one is interested, I will work on it next week. We (webhosting 
> services) really need dynamic configuration per user, without having to 
> reload the whole server.
> 

I'm for it.

Regards,
Patrick

> --
> Neil Dökkalfar

What is about creating a .lighttpdconf file in the document-root of a an 
vhost, and then set a listener on the file to get informed when the file 
has been changed (i mean there's a linux-function for it) and so reload 
ist dynamicly at every change?
the only problem is, when there is no config-file but lighttps could 
create one at registering an vhost.
something like .htaccess is abig need for lighttpd, but not as apache is 
doing this

Email Site wrote:
> I am for the .htaccess module too if it would use the same syntax as 
> Apache :)

Definitely agreed.

It has taken many of us a long time to figure out the voodoo that is 
mod_rewrite, re-learning it in a different syntax wouldn't be nice :)

Guest wrote:
> Email Site wrote:
>> I am for the .htaccess module too if it would use the same syntax as 
>> Apache :)
> 
> Definitely agreed.
> 
> It has taken many of us a long time to figure out the voodoo that is 
> mod_rewrite, re-learning it in a different syntax wouldn't be nice :)

Lighttpd has its own syntax in the config, why should be built another 
syntax into lighttpd? There is already one.

Hmm, what would it take to convince someone to build such a module?
We should create some pot to collect money for the good cause ;).

I don't really care if it takes lighttpd-rules or apache-style rules.

thomas seifert wrote:
> Hmm, what would it take to convince someone to build such a module?
> We should create some pot to collect money for the good cause ;).
> 
> I don't really care if it takes lighttpd-rules or apache-style rules.

No Apache syntax please. LightTPD's is easier. And you will need to make 
a special parser for that case - this doesn't fit lighttpd's philosophy 
about performance and simplicity.

I think the system, if implemented, should work like this:
1). Use two aproaches - files and database.
2). Modify LightTPD so it could reload it's config files without restart 
(that would be nice even if this system is not implemented. Restarting 
whole server due to change of a one param in config file... Not realy 
good).
3). With files it can be made as someone sugested with a listener that 
traks if file was changed. But that maybe not compatitable with Windows 
(if you want that compability). With database it could be made only with 
a notification (like a SIGNAL). Well, for me, it would be OK if I change 
config file and just do like this: lighttpd-reload ./config_file OR 
lighttp-reload --database. Something like this

Psih    wrote:

> 3). With files it can be made as someone sugested with a listener that 
> traks if file was changed. But that maybe not compatitable with Windows 
> (if you want that compability). With database it could be made only with 
> a notification (like a SIGNAL). Well, for me, it would be OK if I change 
> config file and just do like this: lighttpd-reload ./config_file OR 
> lighttp-reload --database. Something like this

Yeah that would be great but AFAIK config-reloading isn't anywhere on 
the todo for the next version(s).
Just found one ticket http://trac.lighttpd.net/trac/ticket/867 which 
didn't raise any interest by the devs :(.

thomas seifert wrote:
> Yeah that would be great but AFAIK config-reloading isn't anywhere on 
> the todo for the next version(s).
> Just found one ticket http://trac.lighttpd.net/trac/ticket/867 which 
> didn't raise any interest by the devs :(.

Bad, too bad. Many users aren't using LightTPD because of that - not 
every http server could be just restarted for a little change in 
configuration.

i would like something like this as well.

right now if any users want to password protect a directory, the server 
admin (me) has to edit the configuration file, or if some elegant system 
is setup to allow users to manage their own confs, the server itself 
still has to be restarted, just for simple http auth on a directory.

Psih    wrote:
> thomas seifert wrote:
>> Yeah that would be great but AFAIK config-reloading isn't anywhere on 
>> the todo for the next version(s).
>> Just found one ticket http://trac.lighttpd.net/trac/ticket/867 which 
>> didn't raise any interest by the devs :(.
> 
> Bad, too bad. Many users aren't using LightTPD because of that - not 
> every http server could be just restarted for a little change in 
> configuration.

Definitely agreed. The question is, just how many posts do we need 
re-iterating the same things over and over again to draw developers 
interest into this subject :)

Guest wrote:
> The question is, just how many posts do we need 
> re-iterating the same things over and over again to draw developers 
> interest into this subject :)

Unfortunately probably too many.
I take a wild guess with telling if no one of us who has knowledge in 
C/C++ and hopefully lighttpd internals will take that challenge then it 
won't be done in the near future.

Guest wrote:
> 
> Definitely agreed. The question is, just how many posts do we need 
> re-iterating the same things over and over again to draw developers 
> interest into this subject :)

Maybe one of the developers could tell the problems of dynamic change of 
the config so that a solution can be discussed.
A modern webserver can not exclude such an option.
This is the reason ehy there'll be never webhoster for example running 
lighttpd

Tobias Petry wrote:
> Guest wrote:
>> 
>> Definitely agreed. The question is, just how many posts do we need 
>> re-iterating the same things over and over again to draw developers 
>> interest into this subject :)
> 
> Maybe one of the developers could tell the problems of dynamic change of 
> the config so that a solution can be discussed.
> A modern webserver can not exclude such an option.
> This is the reason ehy there'll be never webhoster for example running 
> lighttpd

Oh, I got a reply from "darix" while asking on the lighttpd-maillist and 
there seem to be serious problems with the current security concept of 
lighttpd if you want to accomplish that.
I'll just quote him and hope its ok:
---
just throwing in some points from the irc discussion on the same topic
from yesterday:
1. lighttpd has no part that still runs as root
1.a. we cant bind new ports below 1024. therefor we cant guarantee that
     a changed config is really fully reloaded.
1.b. we cant change the runtime user.

jan tried to implement graceful restart but got stopped by at least 1.a.
so we are left with graceful-stop

2. as an user can access every part of the config (even the global one)
   from an included config snippet you always want to verify changes
   done by an user. blindly reloading with user configs can have
   unwanted effects.

you now could argue that lighttpd is not the best choice for multiuser
hosting and you might be right.

but with gracefulstop and a careful config you can at least minimize the
downtime and make config changes less interruptive.
---


thomas