swish/commit

New upstream files

authorJan Wielemaker
Thu Sep 7 13:43:50 2017 +0200
committerJan Wielemaker
Thu Sep 7 13:43:50 2017 +0200
commitcb37381c3bc77940eac65b19eda0184e37f98cfb
treea9c6ea1812dae94902619932f3cc567318bdb002
parent9fec67e917e1703b228e1a0e19c52a542c975474
Diff style: patch stat
diff --git a/config-available/gitty/Hangout.swinb b/config-available/gitty/Hangout.swinb
new file mode 100644
index 0000000..b1d7c62
--- /dev/null
+++ b/config-available/gitty/Hangout.swinb
@@ -0,0 +1,9 @@
+<div class="notebook">
+
+<div class="nb-cell markdown" name="md1">
+### The SWISH hangout room
+
+You can add messages to this room from here or using the __Broadcast to hangout__ option from the __Send__ button on any other file.  In the latter case the message is added to this room with a link to the file.
+</div>
+
+</div>
diff --git a/examples/Rdownload.swinb b/examples/Rdownload.swinb
index 215149c..984b775 100644
--- a/examples/Rdownload.swinb
+++ b/examples/Rdownload.swinb
@@ -1,6 +1,6 @@
 <div class="notebook">
 
-<div class="nb-cell markdown">
+<div class="nb-cell markdown" name="md1">
 # Downloading (graphics) files
 
 The SWISH R interface defines the predicates below for downloading files.  This can be combined with the normal R device manipulation for downloading images.
@@ -13,39 +13,39 @@ The SWISH R interface defines the predicates below for downloading files.  This
 The example illustrates the options using the sine function as defined below.
 </div>
 
-<div class="nb-cell program">
+<div class="nb-cell program" name="p1">
 % Y is sin(X) for X in 0..Max
 sin(Max, X, Y) :-
     between(0, Max, X),
     Y is sin(X*pi/180).
 </div>
 
-<div class="nb-cell markdown">
+<div class="nb-cell markdown" name="md2">
 First, we create an [R dataframe](example/Rdataframe.swinb), load the "ggplot2" library and show the inline SVG.
 </div>
 
-<div class="nb-cell query">
+<div class="nb-cell query" name="q1">
 r_data_frame(df, [x=X,y=Y], sin(360, X, Y)),
 &lt;- library("ggplot2"),
 &lt;- ggplot(data=df, aes(x=x, y=y)) + geom_line().
 </div>
 
-<div class="nb-cell markdown">
-In the next examples we use r_download/0.  This displays the graphics inline, but also provides a download button that allows you to download the SVG to your computer. 
+<div class="nb-cell markdown" name="md3">
+In the next examples we use r_download/0.  This displays the graphics inline, but also provides a download button that allows you to download the SVG to your computer.
 </div>
 
-<div class="nb-cell query">
+<div class="nb-cell query" name="q2">
 r_data_frame(df, [x=X,y=Y], sin(360, X, Y)),
 &lt;- library("ggplot2"),
 &lt;- ggplot(data=df, aes(x=x, y=y)) + geom_line(),
 r_download.
 </div>
 
-<div class="nb-cell markdown">
+<div class="nb-cell markdown" name="md4">
 And finally, we save the graphics to a device (in this example PDF) that we create explicitly.  Note that r_download/0 calls `graphics.off()` before trying to download the generated files.
 </div>
 
-<div class="nb-cell query">
+<div class="nb-cell query" name="q3">
 &lt;- pdf("sine.pdf"),
 r_data_frame(df, [x=X,y=Y], sin(360, X, Y)),
 &lt;- library("ggplot2"),
@@ -53,4 +53,15 @@ r_data_frame(df, [x=X,y=Y], sin(360, X, Y)),
 r_download.
 </div>
 
+<div class="nb-cell markdown" name="md5">
+## Download a file
+
+The example below shows how any file that is saved from R can be made available for download from SWISH.
+</div>
+
+<div class="nb-cell query" data-tabled="true" name="q4">
+&lt;- write.csv(mtcars, file="cars.csv"),
+r_download("cars.csv").
+</div>
+
 </div>
diff --git a/lib/swish/chat.pl b/lib/swish/chat.pl
index b6148c0..cc69e6b 100644
--- a/lib/swish/chat.pl
+++ b/lib/swish/chat.pl
@@ -39,7 +39,8 @@
 	    chat_to_profile/2,		% +ProfileID, :HTML
 	    chat_about/2,		% +DocID, +Message
 
-	    notifications//1		% +Options
+	    notifications//1,		% +Options
+	    broadcast_bell//1		% +Options
 	  ]).
 :- use_module(library(http/hub)).
 :- use_module(library(http/http_dispatch)).
@@ -89,6 +90,11 @@ browsers which in turn may have multiple SWISH windows opened.
      HTTP authentication.
 */
 
+:- multifile swish_config:config/2.
+
+swish_config:config(hangout, 'Hangout.swinb').
+
+
 		 /*******************************
 		 *	ESTABLISH WEBSOCKET	*
 		 *******************************/
@@ -107,13 +113,13 @@ start_chat(Request) :-
 	start_chat(Request, [identity(Identity)]).
 
 start_chat(Request, Options) :-
-	authorized(chat, Options),
+	authorized(chat(open), Options),
 	(   http_in_session(Session)
 	->  CheckLogin = false
 	;   http_open_session(Session, []),
 	    CheckLogin = true
 	),
-	check_flooding,
+	check_flooding(Session),
 	http_parameters(Request,
 			[ avatar(Avatar, [optional(true)]),
 			  nickname(NickName, [optional(true)]),
@@ -124,6 +130,7 @@ start_chat(Request, Options) :-
 			 reconnect(Token),
 			 check_login(CheckLogin)
 		       ], Options, ChatOptions),
+	debug(chat(websocket), 'Accepting (session ~p)', [Session]),
 	http_upgrade_to_websocket(
 	    accept_chat(Session, ChatOptions),
 	    [ guarded(false),
@@ -139,12 +146,12 @@ extend_options([_|T0], Options, T) :-
 	extend_options(T0, Options, T).
 
 
-%!	check_flooding
+%!	check_flooding(+Session)
 %
 %	See whether the client associated with  a session is flooding us
 %	and if so, return a resource error.
 
-check_flooding :-
+check_flooding(_0Session) :-
 	get_time(Now),
 	(   http_session_retract(websocket(Score, Last))
 	->  Passed is Now-Last,
@@ -152,6 +159,8 @@ check_flooding :-
 	;   NewScore = 10,
 	    Passed = 0
 	),
+	debug(chat(flooding), 'Flooding score: ~2f (session ~p)',
+	      [NewScore, _0Session]),
 	http_session_assert(websocket(NewScore, Now)),
 	(   NewScore > 50
 	->  throw(http_reply(resource_error(
@@ -189,7 +198,10 @@ accept_chat_(Session, Options, WebSocket) :-
 	must_succeed(chat_broadcast(UserData.put(_{type:Reason,
 						   visitors:Visitors,
 						   wsid:WSID}))),
-	gc_visitors.
+	gc_visitors,
+	debug(chat(websocket), '~w (session ~p, wsid ~p)',
+	      [Reason, Session, WSID]).
+
 
 reconnect_token(WSID, Token, Options) :-
 	option(reconnect(Token), Options),
@@ -759,8 +771,8 @@ noble_avatar_url(HREF, _Options) :-
 		 *	   BROADCASTING		*
 		 *******************************/
 
-%%	chat_broadcast(+Message)
-%%	chat_broadcast(+Message, +Channel)
+%%	chat_broadcast(+Message) is det.
+%%	chat_broadcast(+Message, +Channel) is det.
 %
 %	Send Message to all known SWISH clients. Message is a valid JSON
 %	object, i.e., a dict or option list.
@@ -789,6 +801,9 @@ subscribed(Channel, WSID) :-
 	subscription(WSID, Channel, _).
 subscribed(Channel, SubChannel, WSID) :-
 	subscription(WSID, Channel, SubChannel).
+subscribed(gitty, SubChannel, WSID) :-
+	swish_config:config(hangout, SubChannel),
+	\+ subscription(WSID, gitty, SubChannel).
 
 
 		 /*******************************
@@ -921,12 +936,23 @@ json_message(Dict, WSID) :-
 	wsid_visitor(WSID, Visitor),
 	update_visitor_data(Visitor, _{name:Name}, 'set-nick-name').
 json_message(Dict, WSID) :-
-	_{type: "chat-message", docid:_} :< Dict, !,
+	_{type: "chat-message", docid:DocID} :< Dict, !,
 	chat_add_user_id(WSID, Dict, Message),
-	chat_relay(Message).
+	(   ws_authorized(chat(post(Message, DocID)), Message.user)
+	->  chat_relay(Message)
+	;   chat_spam(Msg),
+	    hub_send(WSID, json(json{type:forbidden,
+				     action:chat_post,
+				     about:DocID,
+				     message:Msg
+				    }))
+	).
 json_message(Dict, _WSID) :-
 	debug(chat(ignored), 'Ignoring JSON message ~p', [Dict]).
 
+chat_spam("Due to frequent spamming we were forced to limit \c
+	   posting chat messages to users who are logged in.").
+
 dict_file_name(Dict, File) :-
 	atom_string(File, Dict.get(file)).
 
@@ -1175,8 +1201,6 @@ html_string(HTML, String) :-
 		 *	       UI		*
 		 *******************************/
 
-:- multifile swish_config:config/2.
-
 %%	notifications(+Options)//
 %
 %	The  chat  element  is  added  to  the  navbar  and  managed  by
@@ -1195,6 +1219,35 @@ notifications(_Options) -->
 		       ])
 		 ])).
 notifications(_Options) -->
+	[].
+
+%!	broadcast_bell(+Options)//
+%
+%	Adds a bell to indicate central chat messages
+
+broadcast_bell(_Options) -->
+	{ swish_config:config(chat, true),
+	  swish_config:config(hangout, Hangout),
+	  atom_concat('gitty:', Hangout, HangoutID)
+	}, !,
+	html([ a([ class(['dropdown-toggle', 'broadcast-bell']),
+		   'data-toggle'(dropdown)
+		 ],
+		 [ span([ id('broadcast-bell'),
+			  'data-document'(HangoutID)
+			], []),
+		   b(class(caret), [])
+		 ]),
+	       ul([ class(['dropdown-menu', 'pull-right']),
+		    id('chat-menu')
+		  ],
+		  [ li(a('data-action'('chat-shared'),
+			 'Open hangout')),
+		    li(a('data-action'('chat-about-file'),
+			 'Open chat for current file'))
+		  ])
+	     ]).
+broadcast_bell(_Options) -->
 	[].
 
 
diff --git a/lib/swish/chatstore.pl b/lib/swish/chatstore.pl
index a4323de..60520ab 100644
--- a/lib/swish/chatstore.pl
+++ b/lib/swish/chatstore.pl
@@ -35,17 +35,19 @@
 
 :- module(chat_store,
           [ chat_store/1,               % +Message
-            chat_messages/2             % +DocID, -Messages
+            chat_messages/3             % +DocID, -Messages, +Options
           ]).
 :- use_module(library(settings)).
 :- use_module(library(filesex)).
-:- use_module(library(readutil)).
+:- use_module(library(option)).
 :- use_module(library(sha)).
+:- use_module(library(apply)).
 :- use_module(library(http/http_dispatch)).
 :- use_module(library(http/http_parameters)).
 :- use_module(library(http/http_json)).
 
 :- http_handler(swish(chat/messages), chat_messages, [ id(chat_messages) ]).
+:- http_handler(swish(chat/status),   chat_status,   [ id(chat_status)   ]).
 
 :- setting(directory, callable, data(chat),
 	   'The directory for storing chat messages.').
@@ -94,6 +96,15 @@ chat_file(DocID, File) :-
     chat_dir_file(DocID, Dir, File),
     make_directory_path(Dir).
 
+%!  existing_chat_file(+DocID, -File) is semidet.
+%
+%   True when File is the path of   the  file holding chat messages from
+%   DocID.
+
+existing_chat_file(DocID, File) :-
+    chat_dir_file(DocID, _, File),
+    exists_file(File).
+
 %!  chat_store(+Message:dict) is det.
 %
 %   Add a chat message to the chat  store. If `Message.create == false`,
@@ -102,7 +113,8 @@ chat_file(DocID, File) :-
 %   an ongoing chat so we know to which version chat messages refer.
 
 chat_store(Message) :-
-    chat{docid:DocID} :< Message,
+    chat{docid:DocIDS} :< Message,
+    atom_string(DocID, DocIDS),
     chat_file(DocID, File),
     (	del_dict(create, Message, false, Message1)
     ->	exists_file(File)
@@ -111,10 +123,12 @@ chat_store(Message) :-
     !,
     strip_chat(Message1, Message2),
     with_mutex(chat_store,
-               setup_call_cleanup(
-                   open(File, append, Out, [encoding(utf8)]),
-                   format(Out, '~q.~n', [Message2]),
-                   close(Out))).
+               (   setup_call_cleanup(
+                       open(File, append, Out, [encoding(utf8)]),
+                       format(Out, '~q.~n', [Message2]),
+                       close(Out)),
+                   increment_message_count(DocID)
+               )).
 chat_store(_).
 
 %!  strip_chat(_Message0, -Message) is det.
@@ -134,24 +148,120 @@ strip_chat_user(User0, User) :-
 strip_chat_user(User, User).
 
 
-%!  chat_messages(+DocID, -Messages:list) is det.
+%!  chat_messages(+DocID, -Messages:list, +Options) is det.
+%
+%   Get messages associated with DocID.  Options include
 %
-%   Get all messages associated with DocID.
+%     - max(+Max)
+%     Maximum number of messages to retrieve.  Default is 25.
+%     - after(+TimeStamp)
+%     Only get messages after TimeStamp
 
-chat_messages(DocID, Messages) :-
-    chat_dir_file(DocID, _, File),
-    (   exists_file(File)
-    ->  read_file_to_terms(File, Messages, [encoding(utf8)])
+chat_messages(DocID, Messages, Options) :-
+    (   existing_chat_file(DocID, File)
+    ->  read_messages(File, Messages0, Options),
+        filter_old(Messages0, Messages, Options)
     ;   Messages = []
     ).
 
+read_messages(File, Messages, Options) :-
+    setup_call_cleanup(
+        open(File, read, In, [encoding(utf8)]),
+        read_messages_from_stream(In, Messages, Options),
+        close(In)).
+
+read_messages_from_stream(In, Messages, Options) :-
+    option(max(Max), Options, 25),
+    integer(Max),
+    seek(In, 0, eof, _Pos),
+    backskip_lines(In, Max),
+    !,
+    read_terms(In, Messages).
+read_messages_from_stream(In, Messages, _Options) :-
+    seek(In, 0, bof, _NewPos),
+    read_terms(In, Messages).
+
+read_terms(In, Terms) :-
+    read_term(In, H, []),
+    (   H == end_of_file
+    ->  Terms = []
+    ;   Terms = [H|T],
+        read_terms(In, T)
+    ).
+
+backskip_lines(Stream, Lines) :-
+    byte_count(Stream, Here),
+    between(10, 20, X),
+    Start is max(0, Here-(1<<X)),
+    seek(Stream, Start, bof, _NewPos),
+    skip(Stream, 0'\n),
+    line_starts(Stream, Here, Starts),
+    reverse(Starts, RStarts),
+    nth1(Lines, RStarts, LStart),
+    !,
+    seek(Stream, LStart, bof, _).
+
+line_starts(Stream, To, Starts) :-
+    byte_count(Stream, Here),
+    (   Here >= To
+    ->  Starts = []
+    ;   Starts = [Here|T],
+        skip(Stream, 0'\n),
+        line_starts(Stream, To, T)
+    ).
+
+filter_old(Messages0, Messages, Options) :-
+    option(after(After), Options),
+    After > 0,
+    !,
+    include(after(After), Messages0, Messages).
+filter_old(Messages, Messages, _).
+
+after(After, Message) :-
+    is_dict(Message),
+    Message.get(time) > After.
+
+%!  chat_message_count(+DocID, -Count) is det.
+%
+%   Count the number of message stored for   DocID.  This is the same as
+%   the number of lines.
+
+:- dynamic  message_count/2.
+:- volatile message_count/2.
+
+chat_message_count(DocID, Count) :-
+    message_count(DocID, Count),
+    !.
+chat_message_count(DocID, Count) :-
+    count_messages(DocID, Count),
+    asserta(message_count(DocID, Count)).
+
+count_messages(DocID, Count) :-
+    (   existing_chat_file(DocID, File)
+    ->  setup_call_cleanup(
+            open(File, read, In, [encoding(iso_latin_1)]),
+            (   skip(In, 256),
+                line_count(In, Line)
+            ),
+            close(In)),
+        Count is Line - 1
+    ;   Count = 0
+    ).
+
+increment_message_count(DocID) :-
+    clause(message_count(DocID, Count0), _, CRef),
+    !,
+    Count is Count0+1,
+    asserta(message_count(DocID, Count)),
+    erase(CRef).
+increment_message_count(_).
+
 %!  swish_config:chat_count_about(+DocID, -Count)
 %
 %   True when Count is the number of messages about DocID
 
 swish_config:chat_count_about(DocID, Count) :-
-    chat_messages(DocID, Messages),
-    length(Messages, Count).
+    chat_message_count(DocID, Count).
 
 
 		 /*******************************
@@ -164,7 +274,33 @@ swish_config:chat_count_about(DocID, Count) :-
 
 chat_messages(Request) :-
     http_parameters(Request,
-                    [ docid(DocID, [])
+                    [ docid(DocID, []),
+                      max(Max, [nonneg, optional(true)]),
+                      after(After, [number, optional(true)])
                     ]),
-    chat_messages(DocID, Messages),
+    include(ground, [max(Max), after(After)], Options),
+    chat_messages(DocID, Messages, Options),
     reply_json_dict(Messages).
+
+%!  chat_status(+Request)
+%
+%   HTTP handler that returns chat status for document
+
+chat_status(Request) :-
+    http_parameters(Request,
+                    [ docid(DocID, []),
+                      max(Max, [nonneg, optional(true)]),
+                      after(After, [number, optional(true)])
+                    ]),
+    include(ground, [max(Max), after(After)], Options),
+    chat_message_count(DocID, Total),
+    (   Options == []
+    ->  Count = Total
+    ;   chat_messages(DocID, Messages, Options),
+        length(Messages, Count)
+    ),
+    reply_json_dict(
+        json{docid: DocID,
+             total: Total,
+             count: Count
+            }).
diff --git a/lib/swish/page.pl b/lib/swish/page.pl
index 8a18866..e6ad75e 100644
--- a/lib/swish/page.pl
+++ b/lib/swish/page.pl
@@ -144,7 +144,7 @@ swish_reply3(json, Options) :-
 	option(code(Code), Options), !,
 	option(meta(Meta), Options, _{}),
 	option(chat_count(Count), Options, 0),
-	reply_json_dict(json{data:Code, meta:Meta, chats:_{count:Count}}).
+	reply_json_dict(json{data:Code, meta:Meta, chats:_{total:Count}}).
 swish_reply3(_, Options) :-
 	swish_config:reply_page(Options), !.
 swish_reply3(_, Options) :-
@@ -346,7 +346,8 @@ swish_navbar(Options) -->
 			 ul([class([nav, 'navbar-nav', 'navbar-right'])],
 			    [ li(\notifications(Options)),
 			      li(\search_box(Options)),
-			      \li_login_button(Options)
+			      \li_login_button(Options),
+			      li(\broadcast_bell(Options))
 			    ])
 		       ])
 		 ])).
diff --git a/lib/swish/pep.pl b/lib/swish/pep.pl
index cf64821..e96182e 100644
--- a/lib/swish/pep.pl
+++ b/lib/swish/pep.pl
@@ -34,7 +34,8 @@
 */
 
 :- module(swish_pep,
-          [ authorized/2                               % +Request, +Action
+          [ authorized/2,               % +Action, +Options
+            ws_authorized/2             % +Action, +WSID
           ]).
 :- use_module(library(debug)).
 :- use_module(library(option)).
@@ -86,8 +87,10 @@ Examples are:
 %         Update (save) a physical file outside the versioned gitty
 %         store.
 %     * Social options
-%       - chat
+%       - chat(open)
 %         Open websocket chat channel
+%       - chat(post(Message, About))
+%         Post a chat message about a specific topic
 %
 %   @throws http_reply(forbidden(URL)) if the action is not allowed. Can
 %   we generate a JSON error object?
@@ -106,6 +109,23 @@ authorized(Action, Options) :-
         throw(http_reply(forbidden(Path)))
     ).
 
+%!  ws_authorized(+Action, +WSUser) is semidet.
+%
+%   True when WSUser is allowed to  perform   action.  WSUser  is a dict
+%   containing the user info as  provided by chat:chat_add_user_id/3. It
+%   notably has a key `profile_id` if the user is logged on.
+%
+%   @tbd Generalise. Notably, how do we get the identity as
+%   authenticate/2 returns?
+
+ws_authorized(Action, _WSUser) :-
+    var(Action),
+    !,
+    instantiation_error(Action).
+ws_authorized(chat(post(_,_)), WSUser) :-
+    _Profile = WSUser.get(profile_id).
+
+
 :- multifile
     approve/2,
     deny/2.
@@ -130,7 +150,7 @@ approve(file(update(_File, _Meta)), Auth) :-
     user_property(Auth, login(local)).
 approve(run(any, _), Auth) :-
     user_property(Auth, login(local)).
-approve(chat, _).
+approve(chat(open), _).
 
 %!  deny(+Auth, +Id)
 
diff --git a/lib/swish/plugin/profile.pl b/lib/swish/plugin/profile.pl
index 47b4c6a..9f6cc41 100644
--- a/lib/swish/plugin/profile.pl
+++ b/lib/swish/plugin/profile.pl
@@ -226,6 +226,14 @@ swish_config:reply_logged_out(Options) :-
 swish_config:reply_logged_out(_) :-
     broadcast(swish(logout(-))).        % ?
 
+:- listen(swish(logout(http)), cancel_session_profile).
+
+cancel_session_profile :-
+    (   http_in_session(_)
+    ->  forall(http_session_retract(profile_id(ProfileID)),
+               broadcast(swish(logout(ProfileID))))
+    ;   true
+    ).
 
 %!  create_profile(+UserInfo, +IDProvider, -ProfileID)
 %
diff --git a/lib/swish/projection.pl b/lib/swish/projection.pl
index ab83c08..3cc6939 100644
--- a/lib/swish/projection.pl
+++ b/lib/swish/projection.pl
@@ -62,7 +62,9 @@ set.
 
 projection(_).
 
-swish:goal_expansion((projection(Spec),Body), Ordered) :-
+swish:goal_expansion((Projection,Body), Ordered) :-
+    nonvar(Projection),
+    Projection = projection(Spec),
     must_be(list, Spec),
     phrase(order(Spec, Vars), Order),
     Order \== [],
diff --git a/lib/swish/r_swish.pl b/lib/swish/r_swish.pl
index 9d976fb..6c15b35 100644
--- a/lib/swish/r_swish.pl
+++ b/lib/swish/r_swish.pl
@@ -59,7 +59,8 @@ connection library.
 
 :- multifile
 	r_call:r_console/2,
-	r_call:r_display_images/1.
+	r_call:r_display_images/1,
+	r_call:r_console_property/1.
 
 %%	r_call:r_console(+Stream, ?Data)
 %
@@ -75,6 +76,13 @@ send_html(HTML) :-
 	with_output_to(string(HTMlString), print_html(Tokens)),
 	pengine_output(HTMlString).
 
+%!	r_call:r_console_property(?Property)
+%
+%	Relay the size of the console
+
+r_call:r_console_property(size(Rows, Cols)) :-
+	swish:tty_size(Rows, Cols).
+
 %%	r_call:r_display_images(+Images)
 %
 %	Relay   received   images   to   the     SWISH   console   using
diff --git a/lib/swish/storage.pl b/lib/swish/storage.pl
index bf09f21..c3c7d35 100644
--- a/lib/swish/storage.pl
+++ b/lib/swish/storage.pl
@@ -403,7 +403,7 @@ storage_get(raw, Dir, Type, FileOrHash, _Request) :-
 storage_get(json, Dir, Type, FileOrHash, _Request) :-
 	gitty_data_or_default(Dir, Type, FileOrHash, Code, Meta),
 	chat_count(Meta, Count),
-	reply_json_dict(json{data:Code, meta:Meta, chats:_{count:Count}}).
+	reply_json_dict(json{data:Code, meta:Meta, chats:_{total:Count}}).
 storage_get(history(Depth, Includes), Dir, _, File, _Request) :-
 	gitty_history(Dir, File, History, [depth(Depth),includes(Includes)]),
 	reply_json_dict(History).
diff --git a/lib/swish/template_hint.pl b/lib/swish/template_hint.pl
index 8902835..7e49788 100644
--- a/lib/swish/template_hint.pl
+++ b/lib/swish/template_hint.pl
@@ -43,6 +43,7 @@
 :- use_module(library(pldoc/doc_process)).
 :- use_module(library(pldoc/doc_wiki)).
 :- use_module(library(pldoc/doc_modes)).
+:- use_module(library(doc_http)).
 :- use_module(library(http/html_write)).
 :- use_module(library(memfile)).
 :- use_module(library(sgml)).
@@ -67,6 +68,10 @@ SWISH editor.
 @tbd	Dedicated template for the rendering support?
 */
 
+:- if(current_predicate(doc_enable/1)).
+:- initialization(doc_enable(true)).
+:- endif.
+
 %%	visible_predicate_templates(+Module, +Options, -Templates) is det.
 %
 %	True when Templates is a JSON dict holding autocompletion
diff --git a/lib/swish/trace.pl b/lib/swish/trace.pl
index f7fb6cd..5fc4a73 100644
--- a/lib/swish/trace.pl
+++ b/lib/swish/trace.pl
@@ -3,7 +3,7 @@
     Author:        Jan Wielemaker
     E-mail:        J.Wielemaker@vu.nl
     WWW:           http://www.swi-prolog.org
-    Copyright (c)  2015-2016, VU University Amsterdam
+    Copyright (c)  2015-2017, VU University Amsterdam
     All rights reserved.
 
     Redistribution and use in source and binary forms, with or without
@@ -73,7 +73,22 @@ Allow tracing pengine execution under SWISH.
 user:message_hook(trace_mode(_), _, _) :-
 	pengine_self(_), !.
 
+%!	trace_pengines
+%
+%	If true, trace in the browser. If false, use the default tracer.
+%	This allows for debugging  pengine   issues  using the graphical
+%	tracer from the Prolog environment using:
+%
+%	    ?- retractall(swish_trace:trace_pengines).
+%	    ?- tspy(<some predicate>).
+
+:- dynamic
+	trace_pengines/0.
+
+trace_pengines.
+
 user:prolog_trace_interception(Port, Frame, _CHP, Action) :-
+	trace_pengines,
 	pengine_self(Pengine),
 	prolog_frame_attribute(Frame, predicate_indicator, PI),
 	debug(trace, 'HOOK: ~p ~p', [Port, PI]),
@@ -100,6 +115,7 @@ user:prolog_trace_interception(Port, Frame, _CHP, Action) :-
 	trace_action(Reply, Port, Frame, Action), !,
 	debug(trace, 'Action: ~p --> ~p', [Reply, Action]).
 user:prolog_trace_interception(Port, Frame0, _CHP, nodebug) :-
+	trace_pengines,
 	pengine_self(_),
 	prolog_frame_attribute(Frame0, goal, Goal),
 	prolog_frame_attribute(Frame0, level, Depth),
@@ -468,6 +484,7 @@ find_source(Predicate, File, Line) :-
 :- multifile pengines:prepare_goal/3.
 
 pengines:prepare_goal(Goal0, Goal, Options) :-
+	forall(set_screen_property(Options), true),
 	option(breakpoints(Breakpoints), Options),
 	Breakpoints \== [],
 	pengine_self(Pengine),
@@ -475,6 +492,32 @@ pengines:prepare_goal(Goal0, Goal, Options) :-
 	maplist(set_file_breakpoints(Pengine, File, Text), Breakpoints),
 	Goal = (debug, Goal0).
 
+%!	swish:tty_size(-Rows, -Cols)
+%
+%	Emulate obtaining the screen size. Note that the reported number
+%	of columns is the height  of  the   container  as  the height of
+%	answer pane itself is determined by the content.
+
+set_screen_property(Options) :-
+	pengine_self(Pengine),
+	screen_property(Property),
+	option(Property, Options),
+	assertz(Pengine:screen_property(Property)).
+
+screen_property(height(_)).
+screen_property(width(_)).
+screen_property(rows(_)).
+screen_property(cols(_)).
+
+swish:tty_size(Rows, Cols) :-
+	pengine_self(Pengine),
+	Pengine:screen_property(rows(Rows)),
+	Pengine:screen_property(cols(Cols)).
+
+%!	set_file_breakpoints(+Pengine, +File, +Text, +Dict)
+%
+%	Set breakpoints for included files.
+
 set_file_breakpoints(_Pengine, PFile, Text, Dict) :-
 	debug(trace(break), 'Set breakpoints at ~p', [Dict]),
 	_{file:FileS, breakpoints:List} :< Dict,
@@ -490,6 +533,10 @@ set_file_breakpoints(_Pengine, PFile, Text, Dict) :-
 	;   debug(trace(break), 'Not in included source', [])
 	).
 
+%!	set_pengine_breakpoint(+Pengine, +File, +Text, +Dict)
+%
+%	Set breakpoints on the main Pengine source
+
 set_pengine_breakpoint(Owner, File, Text, Line) :-
 	debug(trace(break), 'Try break at ~q:~d', [File, Line]),
 	line_start(Line, Text, Char),
@@ -629,7 +676,7 @@ sandbox:safe_primitive(system:tracing).
 sandbox:safe_primitive(edinburgh:debug).
 sandbox:safe_primitive(system:deterministic(_)).
 sandbox:safe_primitive(swish_trace:residuals(_,_)).
-
+sandbox:safe_primitive(swish:tty_size(_Rows, _Cols)).
 
 		 /*******************************
 		 *	      MESSAGES		*
diff --git a/web/help/about.html b/web/help/about.html
index 6df9eb8..135ed18 100644
--- a/web/help/about.html
+++ b/web/help/about.html
@@ -46,7 +46,7 @@ href="http://www.swi-prolog.org/pldoc/package/cql">CQL</a> to explore
 relational (SQL) databases or <a
 href="http://www.swi-prolog.org/pack/list?p=sparkle">sparkle</a> to
 explore linked data. A <a
-href="http://cliopatria.swi-prolog.org/packs/swish>">ClioPatria
+href="http://cliopatria.swi-prolog.org/packs/swish">ClioPatria
 plugin</a> adds Prolog based exploration of RDF data to ClioPatria.
 </p>
 
diff --git a/web/help/beware.html b/web/help/beware.html
index f919473..985cf41 100644
--- a/web/help/beware.html
+++ b/web/help/beware.html
@@ -76,11 +76,11 @@ again in a couple of months. These are the main plans:
 </ul>
 
 <p>
-Please be patient or contribute to <span style="color:darkblue">SWI</span><span style="color:maroon">SH</span> at <a href="https:github.com/SWI-Prolog/swish.git">GitHub</a>
+Please be patient or contribute to <span style="color:darkblue">SWI</span><span style="color:maroon">SH</span> at <a href="https://github.com/SWI-Prolog/swish.git">GitHub</a>
 
 <div class="github">
-<iframe class="github-btn" src="http://ghbtns.com/github-btn.html?user=SWI-Prolog&amp;repo=swish&amp;type=watch&amp;count=true" width="100" height="20" title="Star on GitHub"></iframe>
-<iframe class="github-btn" src="http://ghbtns.com/github-btn.html?user=SWI-Prolog&amp;repo=swish&amp;type=fork&amp;count=true" width="102" height="20" title="Fork on GitHub"></iframe>
+<iframe class="github-btn" src="https://ghbtns.com/github-btn.html?user=SWI-Prolog&amp;repo=swish&amp;type=watch&amp;count=true" width="100" height="20" title="Star on GitHub"></iframe>
+<iframe class="github-btn" src="https://ghbtns.com/github-btn.html?user=SWI-Prolog&amp;repo=swish&amp;type=fork&amp;count=true" width="102" height="20" title="Fork on GitHub"></iframe>
 </div>
 
 </body>
diff --git a/web/help/chat.html b/web/help/chat.html
index 265967f..9941463 100644
--- a/web/help/chat.html
+++ b/web/help/chat.html
@@ -2,7 +2,7 @@
 
 <html>
   <head>
-  <title>SWISH chat service</title>
+  <title>Chat with SWISH users</title>
   </head>
   <body>
 
@@ -10,16 +10,31 @@
 <p>
 The <span style="color:darkblue">SWI</span><span
 style="color:maroon">SH</span> chat services allows you to chat with
-users about a file. <span style="color:darkblue">SWI</span><span
-style="color:maroon">SH</span> displays an <i>avatar</i> for each user
-with whom you share an open file. If you select <b>File/Chat ...</b> the
-current notebook or program gets a chat window displayed below it,
-preloaded with older chat messages about this file.  A chat message
-may include one or more <b>payload</b> objects that are made visible
-as buttons.  Most payloads are added using the <a class="btn
-btn-xs btn-primary">Send <b class="caret"></b></a> button menu.  Defined
-payloads are:
-</p>
+other <span style="color:darkblue">SWI</span><span
+style="color:maroon">SH</span> users.  Each file acts as a chat room,
+while the <b>Hangout</b> room allows you to attrack attention to the
+file you are working on.  The <b>bell</b> at the top-right shows the
+status of the hangout room and any message sent to the hangout room
+appears briefly below the bell.  The associated <i>dropdown menu</i>
+provides access to both the hangout room and the room associated with
+the currently active file.
+
+<p>
+<span style="color:darkblue">SWI</span><span
+style="color:maroon">SH</span> displays an <b>avatar</b> for each user
+with whom you share an open file to make you aware of other users.
+Operations such as opening, reloading or closing a file are briefly
+indicated to people with whom you share an open file. The <i>avatar</i>
+is taken from your identity provider or <a
+href="https://en.gravatar.com">gravatar.com</a> if you are logged in.
+Otherwise it is a randomly generated avatar that is stored in your
+browser's local store.
+
+<p>
+A chat message may include one or more <b>payload</b> objects that are
+made visible as buttons. Most payloads are added using the <a class="btn
+btn-xs btn-primary">Send <b class="caret"></b></a> button menu. Defined
+payloads are: </p>
 
   <ul>
     <li>The <b>selection</b>.  If you have made a selection on the
@@ -42,15 +57,15 @@ file to <b>ask for support</b>.
 </p>
 
 
-<h3 id="chat-help">The shared chat room</h3>
+<h3 id="chat-help">The hangout room</h3>
 
-<p class="dropup">
-The notebook <a href="/p/Help.swinb">Help.swinb</a> provides a place to
-find buddies. It can be opened from <b>File/Chat help room ...</b> and
-messages may be cross-posted to this shared space using <b>Broadcast to
-help room</b> from the <a class="btn btn-xs btn-primary">Send <b
-class="caret"></b></a> button. The message that appears in the shared
-room contains a link to the file from which it was created.
+<p class="dropup"> The notebook <a
+href="/p/Hangout.swinb">Hangout.swinb</a> provides a place to find
+buddies. Messages may be cross-posted to this shared space using
+<b>Broadcast to hangout</b> from the <a class="btn btn-xs
+btn-primary">Send <b class="caret"></b></a> button. The message that
+appears in the hangout room contains a link to the file from which it was
+created.
 <p>
 
 <h3 id="chat-markdown">Markdown in chat messages</h3>
diff --git a/web/help/hangout.html b/web/help/hangout.html
new file mode 100644
index 0000000..4cc73b0
--- /dev/null
+++ b/web/help/hangout.html
@@ -0,0 +1,30 @@
+<!DOCTYPE HTML>
+
+<html>
+  <head>
+  <title>Using the hangout room</title>
+  </head>
+  <body>
+
+<div class="dropup">
+<p>
+The <b>hangout</b> room is to find buddies.  If you have a question or
+want to start a discussion
+
+<ol>
+  <li>Create a program or notebook by adding a new tab and selecting
+      the appropriate document type and profile.
+  <li>Prepare code and queries in this document.
+  <li>Use <b>Open chat for current file</b> from the top-right <b>bell</b>
+  <li>Create a chat message, briefly explaining what you want.
+  <li>Wait for people to join you at the page and discuss the problem.
+</ol>
+
+<p>
+Please restrict messages placed directly in the hangout to general questions
+about Prolog or this service.  Click the message area again to prepare a
+message.
+</div>
+
+</body>
+</html>
diff --git a/web/help/newchat.html b/web/help/newchat.html
new file mode 100644
index 0000000..24a6fee
--- /dev/null
+++ b/web/help/newchat.html
@@ -0,0 +1,25 @@
+<!DOCTYPE HTML>
+
+<html>
+  <head>
+  <title>Chat about a file</title>
+  </head>
+  <body>
+
+<div class="dropup">
+<p>
+You are starting a chat about a file.  The first message you enter will be
+broadcasted to the <b>hangout</b> including a link to this file.  So please
+
+<ol>
+  <li>Do not include source code, etc.  People will click on the link
+      and find your code here.
+  <li>You may wish to add your current query using the <a class="btn
+      btn-xs btn-primary">Send <b class="caret"></b></a>.
+  <li>Include a descriptive test such as <i>Why do I get <b>instantiation
+      error</b>?</i>
+</ol>
+</div>
+
+</body>
+</html>