sorokin.oleg.edu/learnharness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

repo: public housekeeping — license, product name, docs diet

Browse Source 
- LICENSE: AGPL-3.0-only (holder Amos Blomqvist), package.json gains "license".
- Product/package name unified: package.json "name" is now "learnharness"
  (matches the GitHub repo); electron-builder productName/appId already
  were LearnHarness/com.learnharness.LearnHarness.
- Deterministic userData: app.setName("LearnHarness") early in main.cjs
  — Application Support dir is exactly "LearnHarness" in dev and
  packaged runs (packaged Electron would otherwise derive it from the
  package name). Dev shell becomes "LearnHarness-dev". Pre-release
  rename: zero external users, no migration code; the local test dir
  was moved by hand.
- docs/ diet: nine vision-*.md splinters, the completed pi-ai migration
  plan, and research/ notes leave the tree (retained privately in the
  git-ignored memory; full history stays in git). Public docs remain:
  architecture, vision, plan, probe-teach-split, pi-engine-swap,
  keyboard, teaching-methodology (+ release-0.3 until the ship).

Gates: 1489/1489, typecheck, lint.
This commit is contained in:
amosblomqvist
2026-10-10 11:20:20 +03:00
parent 26bc305d66
commit 0a7d12c364
15 changed files with 671 additions and 1393 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LICENSE
+661
View File
@@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<https://www.gnu.org/licenses/>.
docs/PI-NATIVE-MIGRATION-PLAN.md
-624
View File
@@ -1,624 +0,0 @@
# Pi-native architecture migration plan
Recon wave 2026-09-26 · READ-ONLY deliverable · `feat/platform` · untracked until owner review
Evidence: wiretap `/tmp/lh-wire/`, 11 controlled OpenRouter replays (`/tmp/rp-*.json/.out`), pi-ai + pi-agent-core dist sources, pi's own session files under `~/.pi/agent/sessions/`.
---
## 0. Executive summary
- **Two banked facts are overturned by the wiretap evidence** (§1). The tool-stripping mystery was model confabulation — the registry was complete on the wire. The truncation is an upstream (OpenInference) bug triggered by multi-tool-result history volume, not by our serialized shape.
- Five custom seams mapped against pi-native equivalents (§2). Each has a native replacement; the getApiKey fix (76051d0) is the template: delete the custom seam, use the library path.
- What genuinely has no native equivalent is small and enumerable (§3): question-tool UX, tool toggles/phases, jobs, prompt overrides, our reattach/persistence infrastructure.
- Proposed risk order (§4) and test strategy (§5). Seams touching `src/agent` are flagged for Teacher's review per band rules.
---
## 1. Corrections to banked facts (evidence first)
### 1.1 TOOL-SET STRIPPING — SOLVED: the model confabulated; no seam varies the registry
- Wiretap captures **7 and 9 are chat_muhb8l00's two exact turns** (timestamps 18:44:47 / 18:48:10 match the record created 18:44:41Z). **Both carried the full 11-tool registry** — `quiz, ask_user_question, read, ls, find, grep, research, pdf, list_sessions, list_propositions, probe_done` — including the graph tools the model claimed not to have.
- The response-side capture shows the same request streaming the "no graph tools in this session… the harness is stripped to the two question tools" answer with `finish_reason: stop`. The claim is false; z-ai/glm-5.3-flash (relayed via OpenRouter upstreams Relace/OpenInference) confabulated a toolset, likely trusting prose descriptions over the actual `tools` array.
- Census of every "missing tool" claim across all master chat records (`data/chats/*.json`):
- `chat_mtxsy1lq` (09-12), `chat_mucnl5j3`/`chat_mucnyjvu` (09-22): honest — probe-phase (no graph tools yet) and/or research correctly absent (no search key → `searchAvailable` false).
- `chat_mu73galx` (09-18): the closed "no sub-agents" scare (probe-phase prober, teacher tools are teacher-only).
- `chat_mufpa030` (09-24): model accurately enumerated its full toolkit; quiz worked.
- **"Another session reportedly lacked quiz entirely": not found in any record** (regex census over all chats).
- Verdict: `tools.ts` gating and `chat-handler`'s ctx passing work as designed. No `src/agent` change required for this mystery (flagged for Teacher anyway since tools.ts is in that band). Mitigation is model-facing (stronger system-prompt tool enumeration, or dropping flash for tool-heavy phases) — an owner decision, not a seam fix.
### 1.2 HISTORY-SERIALIZATION TRUNCATION — REFINED: upstream bug on multi-tool-result histories; our shape is not the trigger
11 controlled replays today against `z-ai/glm-5.3-flash` via OpenRouter (all served by the **OpenInference** upstream; the original truncating request `/tmp/replay.json` = wiretap capture 2, the ML session):
| variant | change vs. truncating request | prompt_tokens | outcome |
|---|---|---|---|
| base | verbatim | 7006 | **TRUNC** (4 chunks, ~22 chars reasoning, billed 258 completion, false `finish: stop`) |
| A | assistant content cleaned (no glued thinking), parallel calls kept | 6803 | **TRUNC** |
| B | parallel calls split into two sequential assistant turns | 7017 | **TRUNC** |
| G | `reasoning` param removed | 7006 | **TRUNC** |
| H | `max_completion_tokens` 131072→8192 | 7006 | **TRUNC** |
| D | both tool results shortened to ~30 chars | 5982 | works (11 tool calls) |
| P | two results at 100+100 chars | 6037 | works |
| L | two results at 300+300 chars | 6201 | **TRUNC** |
| K | two results at 700+700 chars | 6564 | **TRUNC** |
| J | big result (1564) + tiny (30) | 6658 | **TRUNC** |
| E | single result (1564) | 6626 | works |
| F | single result (802) | 6296 | works |
| C9 | capture 9 verbatim (the request that streamed the confabulation live) | 6146 | works (332 chars) |
| I | single result padded to 2764 chars | 7347 | works (4 tool calls) |
Readings:
- **Not the glued-thinking content** (A truncates clean), **not parallelism** (B truncates sequential), **not prompt size** (I streams 7347 tokens; L truncates at 6201), **not the reasoning param or completion cap** (G, H).
- **Trigger: ≥2 tool-result messages with combined content ≥ ~200–600 chars** (P works at 200, L truncates at 600, J truncates with 1564+30). A single tool result of any tested size (≤2764 chars) streams fine. Direct Anthropic/OpenAI and non-flash GLM tolerate everything.
- The failure mode is consistent: a sliver of reasoning streams, 150–260 completion tokens are billed, `finish_reason: stop` arrives with ~zero content — a silent upstream truncation, stable and deterministic 4+ hours after the live capture.
- **Consequence for the plan:** pi-native serialization would NOT have prevented this — tool results are byte-identical on the wire either way. The live bug needs provider-side action (report to OpenRouter; or pin/avoid the flaky upstream for flash; short-term: route flash traffic elsewhere). The migration below is still correct, for the reasons in §1.3 — but it is not the truncation fix, and saying otherwise would be false comfort.
### 1.3 What IS wrong with our serialization (the honest indictment of record-adapter)
- `src/server/pi/record-adapter.ts:161-163` stamps every replayed assistant message `provider: "openrouter", model: "replayed"` → `transformMessages` (`pi-ai/dist/api/transform-messages.js:66`) sees `isSameModel === false` for **every** replayed turn, forever.
- Cross-model path converts thinking blocks to plain text (`transform-messages.js:78-81`), which `openai-completions.js` then glues into the assistant content string (no separator — capture 2's content is literally `"<thinking>…question.<text>There's already…"`).
- `thinkingSignature` cannot survive the UIMessage round-trip (records persist text only), so replayed turns never restore `reasoning_details`. pi's own sessions do: every assistant line in `~/.pi/agent/sessions/**/*.jsonl` carries full `provider/model/api/usage/stopReason` + `thinkingSignature`, which is why pi's replay stays same-model and signature-faithful.
- Net: every history turn we send diverges from what pi itself would send — discarded reasoning state, cache-hostile, and any provider quirk about reasoning-in-content becomes ours to keep debugging.
---
## 2. Seam-by-seam map: custom remnant → pi-native replacement
### Seam 1 — Message history persistence & replay
- **Now:** `LHChatRecord` stores Vercel-SDK-shaped UIMessages (part states `input-available`/`output-available`, `data-*` UI parts); `src/server/pi/record-adapter.ts` (`recordToPiMessages`, 189 lines) hand-rebuilds pi-ai `Message[]` per turn; marker-only user messages flatten to canonical text; dangling tool calls get synthesized `INTERRUPTED` results; chat-handler rebuilds history on every POST.
- **Native:** pi persists append-only JSONL records of full pi-ai messages (verified in pi's own session files); `Agent` accepts `initialState.messages` (`pi-agent-core/dist/agent.js:21-45`); custom message types ride `CustomAgentMessages` declaration-merging (`types.d.ts:269-283`) and are filtered from the LLM view by the default `convertToLlm` (`agent.js:8-10`) or mapped in a `transformContext` hook (`agent-loop.js:179-183`).
- **Replace with:** store pi `AgentMessage[]` natively in the chat record (custom messages for plan blocks/artifacts/verdict markers instead of `data-*` parts); replay = `initialState.messages` from the store; dangling-call synthesis and marker flattening move into one `convertToLlm`/`transformContext` hook — one place, tested once.
- **Delete:** `record-adapter.ts` wholesale; the UIMessage part-state model as the history format (UI gets a transcript view model instead).
- **Kills:** the `model: "replayed"` stamp, thinking-glue, lost signatures (§1.3) — replayed turns become same-model, signature-faithful, cache-friendly.
### Seam 2 — Tool definition wrapper
- **Now:** `src/agent/lh-tool.ts` (244 lines): `LhTool {name, description, io: zod, execute(args, ctx), clientQuestion}`; `toPiTool` bridges via `z.toJSONSchema`, re-runs zod validation inside `execute` so `superRefine` failures throw `ToolInputError`; `TOOL_TIMEOUT_MS` 30s + 250ms abort grace; `clientQuestion` → `executionMode: "sequential"`.
- **Native:** `AgentTool` (`pi-agent-core/dist/types.d.ts:340`): required `label`, TypeBox `parameters`, optional `prepareArguments` shim (raw-arg rescue — replaces our re-validation hack), `execute(toolCallId, params, signal, onUpdate)`, per-tool `executionMode`, `replay` policy. pi's own tools are `ToolDefinition`s (`pi-coding-agent/dist/core/extensions/types.d.ts:344`: `promptSnippet`, `promptGuidelines`, `renderCall`, `renderResult`) wrapped by `wrapToolDefinition` (`core/tools/tool-definition-wrapper.js`; reference impl `core/tools/read.js`: TypeBox `Type.Object` schema + snippet/guidelines).
- **Replace with:** TypeBox-schema ToolDefinitions; `prepareArguments` for arg rescue; timeout stays inside `execute` (it's ours to keep, but as execute internals, not a wrapper layer).
- **Delete:** `LhTool`, `toPiTool`, the zod→JSON bridge, the `clientQuestion` flag (`executionMode: "sequential"` already expresses it).
- **Teacher flag:** `src/agent` band.
### Seam 3 — Tool registration gates
- **Now:** `src/agent/tools.ts` (244 lines): prober/teacher registries, `filterEnabledTools` (drops `enabled[name] === false`), `probe_done` re-added always-on, `research` gated on `searchAvailable`, graph tools gated on `ctx.graphStore`, quiz schema differs per phase.
- **Native:** registration is plain array assembly (`wrapToolDefinitions`); pi has no gating concept because gating is app policy.
- **Verdict:** the gating logic itself is app policy, works (§1.1 proved no stripping), and stays — but it assembles ToolDefinitions from seam 2 instead of LhTools. `filterEnabledTools` + toggles map unchanged onto the new shape.
- **Teacher flag:** `src/agent` band.
### Seam 4 — Question tools & turn stopping
- **Now:** `src/agent/teacher-agent.ts` `makeStopper` ends the run on a cleanly-resolved question tool call (maxTurns 20 default); question tools `execute` returns a synthetic `'Question posed'` placeholder never persisted; the client renders the question and the learner's answer arrives as the next user message.
- **Native:** `AgentToolResult.terminate: true` — `agent-loop.js:385` `shouldTerminateToolBatch` stops the loop after a batch whose results all terminate; `types.d.ts:237/330` document the flag; `shouldStopAfterTurn` (`agent-loop.js:154`) covers cross-batch policy.
- **Replace with:** quiz/ask execute → `{ content, terminate: true }`. Delete `makeStopper`. The pending-question UX (client rendering, answer flow) is app-level and stays as-is.
- **This is the cleanest example of the whole wave:** our stopper re-implements a library flag.
### Seam 5 — Agent construction & steering
- **Now:** stateless rebuild per POST (`teacher-agent.ts` — fresh Agent per turn, `composeInstructions` + `effectivePromptSegments` per turn, maxTurns); steering via `claimSteers` coalescing + the wake pipeline's deferred turn.
- **Native:** stateful `Agent` (`agent.js`): `initialState`, `subscribe` listeners awaited in settlement, `steer()`/`followUp()` queues (`one-at-a-time`/`all` — `steer()` is literally our wake-message semantics), `sessionId` forwarded for cache-aware providers, `thinkingBudgets`, per-agent `toolExecution` default.
- **Replace with (staged):** keep stateless rebuild first, but feed `initialState.messages` from the native store (seam 1) — same wire, smaller step. A stateful per-chat Agent registry (mirroring `turn-runs.ts`) with `steer()` for the wake path is a later, optional consolidation.
- **Keep:** `composeInstructions`/prompt overrides (app feature), write-ahead persistence, turn gates.
- **Teacher flag:** `src/agent` band.
### Seam 6 — SSE / streaming bridge
- **Now:** `src/server/pi/ui-stream.ts` (423 lines): `runPiTurn` runs the loop, keeps a memory-only chunk log + subscriber set (background-turn wave 00a92e4), and hand-maps pi `AgentEvent`s into the Vercel `UIMessageStream` chunk dialect (text/thinking/toolcall mapping, question-tool output suppression, input gating); `[DONE]` terminal.
- **Native:** pi events are already a typed stream (`AgentEvent`: `agent_start`/`message_update`/`turn_start`/`turn_end`/`agent_end`, carrying partial+final messages; `pi-ai` `AssistantMessageEventStream` underneath); pi's own client binds via transcript snapshots + event subscriptions (`pi-coding-agent/dist/client/remote-session.js`).
- **Replace with:** thin passthrough — serialize pi `AgentEvent`s (or message_update partials) directly instead of remapping into UIMessage chunks; keep our subscriber-log/reattach/abort machinery (that's the 00a92e4 background-turn layer and it's good); the question-tool suppression hacks die naturally because native events carry tool-call state honestly.
- **Cost:** wire dialect changes → **UI stream coordination required** (client half dispatch after merge, same as previous waves).
- **Delete:** the event-mapping switch in `ui-stream.ts` (the log/subscriber/abort halves stay).
### Already native (the template)
- `getApiKey` (fixed 76051d0): the deleted hook is exactly this wave's pattern — custom seam removed, pi-ai's native key resolution used. Key-presence gating stays app-side at the turn gate.
---
## 3. What genuinely needs custom (no native equivalent — keep, but express natively)
1. **Question tools awaiting human answers** — closest native is `terminate: true` + app-level answer flow (the "pause the world for a human" semantics is ours).
2. **Tool toggles / phases / graphTools gating** — app policy at registration time (seam 3).
3. **Jobs + extraction pipeline** — outside the agent loop entirely; untouched by this wave.
4. **Prompt override switches** (`composeInstructions`/`effectivePromptSegments`) — app feature; stays.
5. **Write-ahead persistence, background turns, reattach subscribers, remote abort** — our infrastructure from waves 00a92e4/4d694b3; stays.
---
## 4. Risk order
1. **Wave A — tools + stopping (seams 2+3+4):** smallest blast radius; deletes lh-tool + makeStopper; gates unchanged; UI unaffected (tool names/wire identical).
2. **Wave B — native history store + replay (seam 1):** biggest win (wire fidelity, signatures, cache, kills §1.3), biggest risk (record migration for existing chats). Needs a one-time record migration + golden-master diff (§5) before cutover.
3. **Wave C — stateful agents (seam 5, optional):** only after B settles.
4. **Wave D — SSE dialect (seam 6):** wire contract change; dispatch UI half after merge.
5. **Independent, immediate:** the §1.2 truncation is an upstream bug — report to OpenRouter / route-pin away from the flaky upstream for `glm-5.3-flash`. Not gated on any wave.
---
## 5. Test strategy
- **Wire-shape pin (kills §1.3 forever):** replay a stored same-model session through `convertMessages` and assert the assistant history carries `reasoning_details`/`thinkingSignature` preserved (fails today on `model: "replayed"`).
- **Terminate pins:** question tool result `terminate: true` ends the loop natively (replaces `makeStopper` pins; mirror the existing chat-flow scripted-teacher tests).
- **Registry census pin:** faux-provider wiretap-style test asserting the full registry (11 tools incl. graph tools) is present on the wire for both phases regardless of toggles-off state — pins §1.1 shut.
- **Golden-master for Wave B:** for every stored chat, diff old adapter output vs native replay output message-by-message before cutover; migrate one throwaway copy of master `data/chats` first (established one-way copy rule).
- **Per-wave gates unchanged:** typecheck / 950+ tests / lint / build; live smoke on PORT=3004 (faux + real provider), reattach and abort paths re-verified after Wave D.
---
## 6. Open owner decisions
1. Flash-model policy for tool-heavy phases (confabulation §1.1 + truncation §1.2 both bite `z-ai/glm-5.3-flash` via OpenRouter): route-pin, provider-pin, or accept + prompt-mitigate.
2. Whether to report the OpenInference multi-tool-result truncation to OpenRouter (I can package the 15 replay files as a repro bundle).
3. Wave B record migration: in-place upgrade vs new-record-format-version with lazy dual-read (settings-delta-wave precedent).
4. Wave D timing (UI stream coordination).
---
# Wave B planning — native history store + replay (2026-09-26 recon, post-Wave-A)
Wave A landed (93b59cd): tools/stopping are pi-native. Wave B = seam 1 — the
record becomes pi-native history; `record-adapter.ts` dies. The owner's
standing rulings already settle the plan's §6.3 (record migration): **zero
backwards compat, all session data wiped, no migration** — so the golden-master
cutover is reduced to a fresh-format smoke, not a data project.
## What Wave B replaces today (the seam, precisely)
- `src/server/pi/record-adapter.ts` (189 lines): record→pi-ai rebuild per turn;
the `model: "replayed"` stamp (breaks `isSameModel` forever — §1.3), thinking
glued into content, `thinkingSignature` lost (records persist text only),
verdict/handoff marker flattening (`systemTextOf`), dangling-call
INTERRUPTED synthesis.
- `chat-handler`: write-ahead persist of the CLIENT's full UIMessage list (the
client owns the transcript between turns — useChat resend + addToolOutput),
`recordToPiMessages(outgoing)` at agent build, probe_done detection over
UIMessage parts.
- `ui-stream.runPiTurn`: assembles UIMessage parts from pi events; that
assembly is BOTH the wire dialect (Wave D's concern) AND the persist target.
- `turn-service`: `withMessageIds` (SDK client-side-id patch), plan-block
attachment onto UIMessage parts.
## Consumer census (who reads record.messages as UIMessage part shapes)
Server/core: phases (`phaseAt3` — tool-probe_done part state), pending
(`findPendingToolCall`), plan-blocks (`withPlanBlocks`), coverage
(`coverageOf`), graph extraction sources (`toolParts`), extraction
(`messageViewLines`), store summaries (phase + pendingQuestion),
`/api/chats/[id]` GET, jobs.ts (steers as data-part messages; `claimSteers`
over the POST body). Client: `useChat` POSTs the full list, renders every
part type, `mergeAdopted`, addToolOutput answers, artifact/plan/abort cards.
**None of these need the STORAGE to be UIMessage — they need a UIMessage-SHAPED
VIEW.** That distinction is Wave B's core move.
## The design: server-owned native record + one-way projection
1. **`LHChatRecord.messages: AgentMessage[]`** (pi-ai `UserMessage |
AssistantMessage | ToolResultMessage | CustomMessage`). The agent's own
final messages persist at `message_end` — provider/model/usage/stopReason/
thinkingSignature stamped by the loop itself (what pi persists in its own
sessions). §1.3 dies by construction: replayed turns are byte-faithful,
same-model, signature-preserving; replay = `state.messages` assignment,
zero conversion.
2. **Custom messages for our non-LLM content** (pi's `CustomMessage`: role
"custom", customType, content, display, details): plan-block,
subagent-artifact, subagent-aborted, question-answered markers. The
verdict/probe-handoff steers persist AS custom messages whose CONTENT is
the canonical model-facing text (verdictText / HANDOFF_TEXT written once
at append) — `systemTextOf` flattening disappears; `convertToLlm` default
passes content through, UI-only types filter via details/customType.
3. **One-way projection `AgentMessage[] → UIMessage[]`** (new
`src/server/pi/project.ts`): serves `/api/chats/[id]` GET, the summary
scans (phase/pending/coverage/plan-blocks/graph/extraction keep their
existing UIMessage code — they read the projection), and the client until
Wave D. The lossy DIRECTION (UIMessage→pi) is deleted; the projection is
Wave D's deletion target.
4. **Question answers are native tool results (owner ruling, 2026-09-26).**
The history stays a NORMAL agent history: quiz toolCall followed by its
toolResult — the process just stops in between. The placeholder result
is an in-memory loop device only (terminate:true ends the turn); it is
never persisted, exactly as today's wire suppression. On answer, the
server appends a native ToolResultMessage whose content is the same
encoding the adapter produces today (the model-facing wire is
byte-shape-identical to current behavior). The record carries the
dangling call only in the pose→answer window, where no turn can run
(the question gate holds steers), and the INTERRUPTED synthesis survives
solely for the abort case — same as today. pi's own resume path is the
mechanism: assign state.messages (record ending at the toolResult),
continue().
5. **Turn input contract**: the POST body's new-content DELTA (new user text,
answers, markers) is appended natively; the full-list body stays for now
(client unchanged — see fork 3).
## Forks — ALL RESOLVED (owner rulings, 2026-09-26)
1. **Answer semantics — normal agent history.** The answer is the tool
call's toolResult, appended natively at answer time; the placeholder is
an in-memory loop device only (terminate), never persisted. No custom
answer markers. Verified against pi's source: the loop always pairs
calls with results in a RUN, but the record may end at the assistant
message (no turn can run in the pose→answer window — the question gate
holds steers); pi's own resume path (assign state.messages, continue()
from the toolResult) is the mechanism. The model-facing wire is
byte-shape-identical to today.
2. **Storage shape — `AgentMessage[]` inside LHChatRecord JSON.** No new
dependencies; metadata stays in the one atomic tmp+rename file. (pi
SessionManager JSONL rejected: pi-coding-agent is a 139MB harness dep
and its tree/branching/compaction machinery is unused here.)
3. **POST contract — input events NOW, in this pass** (owner overrode the
server-side-delta recommendation): POST /api/chat carries
`{chatId, input}` where input is one of:
`{type:"text", text, note?}` (composer send; note rides the pending
verdict interplay),
`{type:"answer", toolCallId, answer:{values,other,text,dontKnow,note},
dismissed?}` (quiz/ask answer or dismissal — graded client-side as
today, the structured verdict becomes the native toolResult content),
`{type:"verdict", accepted, note?}` (plan verdict).
Consequences: the client's 409 merge-retry DIES (nothing to merge — a
409 means wait-and-retry the same idempotent event); claimSteers/
write-ahead of full lists dies; the wake pipeline's internal turns
(setTurnRunner) and jobs steers append natively server-side; useChat
stays as the RENDERER (message list = projected record + stream
assembly — receive side unchanged until Wave D); sends go through a
thin client session-client: POST the event, read the status, cancel
the 200 body (a view may detach freely — the turn runs on), then
attach via the standard resume path. Client touches: chat-transport
send half, chat.tsx call sites (composer, onSubmitAnswer, verdict,
dismiss). Receive half (SSE dialect, part rendering) stays Wave D.
4. **Raw vs validated tool args — resolved by design**: persist pi's own
AssistantMessage verbatim (what message_end carries). That means RAW
streamed args — byte-faithful replay, identical to what pi persists in
its own sessions, cache-parity with the original emission. The dock's
parse seam already tolerates raw (Wave A boundaryParse fix); no other
record consumer reads tool-call args.
5. **Abort semantics — folded into 1**: persist partial (current
behavior); the record ends at the dangling call for the abort case and
the INTERRUPTED synthesis at replay covers it (same as today).
## Test strategy
- Wire-shape pin (§5's oldest promise, now trivial): persist a scripted
session's agent messages, rebuild the agent, `convertMessages` → assert
`provider`/`model` real (not "replayed"), `thinkingSignature` present.
- Projection parity: every existing UIMessage consumer test (988 suite) runs
against the projection unchanged — the suite IS the golden master.
- Question flow: pose → answer-as-marker → next turn's model view carries the
verdict; abort mid-question → resume works (kills the synthesis tests).
- Live smoke: real turn + reattach + remote abort on PORT=3004.
## Blast radius (files)
Delete: record-adapter.ts; chat-transport's retryingFetch/merge logic.
New: project.ts (AgentMessage[]→UIMessage projection), input-events
contract (core/contracts), client session-client send module.
Rewrite: chat-handler (build = state.messages from record; input-event
append; probe_done detection over native messages), ui-stream (part
assembly becomes WIRE-ONLY; the record write comes from the agent's own
messages at message_end), turn-service (native appends replace
withMessageIds/withPlanBlocks part mutation; plan-blocks/artifact steers
become CustomMessages), store/chats + core scanners (phaseAt3,
findPendingToolCall, coverage, graph sources — read the projection,
mechanical), jobs.ts (steers append native CustomMessages; setTurnRunner
internal path), client send call sites (composer, onSubmitAnswer, verdict,
dismissal — chat-transport send half). useChat's RECEIVE half and all part
renderers untouched (Wave D). Agent band untouched (Wave A's). Wave C
(steer() consolidation) stays deferred.
---
# Wave D planning — the native wire + client rewrite (2026-09-26 recon, post-Wave-B)
Wave B landed the native record (LhMessage[], input events, projection,
record-adapter death). Wave D = seam 6 + the client half — the wire becomes a
verbatim passthrough of pi's own AgentEvents, the Vercel AI SDK leaves the
codebase entirely, and every projection consumer reads native. Wave C (stateful
agents) stays optional and deferred (owner, 2026-09-26).
## What Wave D replaces (the seam, precisely)
- `ui-stream.ts`'s translation switch (~250 lines): the AgentEvent →
UIMessageChunk remap (text/reasoning part assembly + id minting,
toolInputs/newToolParts bookkeeping, input-available validation gating,
question-output suppression, finishReason mapping, the start/messageId
continuation rule). The log/subscriber/abort/settle machinery STAYS.
- The Vercel SDK client: `ai` + `@ai-sdk/react` packages — `useChat`
(messages/status/sendMessage/addToolOutput/setMessages/error/stop/
clearError/resumeStream), the `DefaultChatTransport` subclass
(chat-transport.ts: send-half prepareSendMessagesRequest, receive-half
reconnectToStream adapters — settled terminal, continuation-seam id
rewrite, SSE head re-injection), `lastAssistantMessageIsCompleteWithToolCalls`.
- `core/projection.ts` + `core/ui-message.ts` (TeacherUIMessage,
mergeAdopted): deleted. The grouping canon moves CLIENT-SIDE (fork 1).
- The projection's readers go native-read: phases, pending, coverage,
graph toolParts, plan-blocks snapshot, extraction view, store summaries,
GET /api/chats/[id], app/chat/[id]/page.tsx.
## Forks — ALL RESOLVED (owner rulings, 2026-09-26)
1. **The client's view state — today's exact look is the golden master.**
Everything the agent produces renders as ONE continuous document per run:
step separators exactly as now, artifact/answered cards inline, the
teacher's reaction continuing below an answered quiz. Implemented by a
pure client-side grouping function (the projection's canon, app-owned,
zero `ai` types) over (record + live turn events). Raw per-native-message
block rendering was rejected (visible layout change).
2. **Question tools on the wire — forward everything.** No server-side
suppression: the quiz's closing `tool_execution_end` (the in-memory
"Question posed" bookkeeping result) flows to the client, which ignores
it (question tools are client-owned — the rule lives where the dock
lives). The server is a pure passthrough with zero special cases. The
placeholder still never persists, never reaches the model, never renders.
3. **Extraction unit — the message, with one inseparability rule.** The
coverage/fingerprint unit is the native message; the batcher's one rule:
a toolCall message and its toolResult message(s) are ALWAYS grouped into
the same chunk (covered together, read together). A fresh answer's
result pulls its asking assistant message into the chunk as context.
Fingerprints reset to the new unit once (scratch data, accepted).
## The design
### Server: verbatim AgentEvent passthrough (ui-stream.ts rewrite)
- The per-turn log stores AgentEvents verbatim; `data: {…}\n\n` lines, one
JSON event each, `data: [DONE]\n\n` terminal. Every pi event type forwards
as-is: agent_start, message_start/update/end, tool_execution_start/update/
end, turn_start, turn_end, agent_end. `message_update` carries the FULL
partial message (`partial: AssistantMessage` on every AssistantMessageEvent)
— the client REPLACES the streaming assistant message; no delta assembly.
- App-level events kept (our protocol, not pi's): the reattach terminal
`{type:"turn-settled"}` and a loop-level error event (agent.continue()
throws outside the event stream; mapped through the handler's onError
unmasking policy). Question suppression: NONE (fork 2).
- The start/messageId continuation seam DIES — the client's grouping canon
handles continuation (an answered-quiz continuation turn appends into the
same group by construction).
- piTurnResponse headers, POST /api/chat response, GET stream route: all
unchanged (same subscriber shape, same terminals).
### Client: the session hook + the view model (the SDK dies)
- **View model** (new client-safe pure module): the grouping canon verbatim
from projection.ts — consecutive assistant+toolResult messages bounded by
user/marker messages form one assistant view unit (step separators at
turn boundaries); toolResults attach to their call's block (backward
toolCallId search — answered cards, error cards, artifact outputs);
marker customs (verdict, probe-handoff, steer cards) project to their
current render shapes; plan-block markers splice after the last successful
plan_init result. Input: (record base + live turn messages). Golden
master: today's UI, byte-for-byte.
- **Session hook** (new, replaces useChat): owns view messages, status
(ready/submitted/streaming vocabulary KEPT — chat.tsx's busy/poll gates
transplant mechanically), error, and the channels:
- `send(input: ChatInput)` — POST {chatId, input} with the 409 identity
retry (retryingFetch survives chat-transport's death); the pendingInput
stash + loud-throw contract stays, enforced in the hook.
- `answer(toolCallId, answer)` — optimistic local result append + send
(the addToolOutput replacement; same UX: instant graded card).
- `attach()` — GET /api/chats/[id]/stream → the same SSE event parser;
turn-settled → syncFromServer (the settled terminal's contract).
- abort channel unchanged (POST /api/activity/abort); cancel detaches
the view only (the background-turn ruling is untouched).
- **SSE parser**: data lines → view updates (message_update replaces the
streaming assistant; message_end finalizes; turn_end appends results;
agent_end → settle → syncFromServer; error event → the banner).
- **Mid-turn record adoption** (the mergeAdopted lesson, native): steers/
artifacts persist into the record while a turn runs; on the jobs-poll
nudge the hook re-reads the record and rebases the view when it is
append-only below the live tail (else defers to full sync at idle).
- **Renderers**: chat.tsx's switch moves from UIMessage part types to view
blocks (text/reasoning/toolCall-with-result) — visually identical.
ai-elements' `import type { UIMessage}` swaps to view types; streamdown
stays (independent library).
### Scanners go native (projection deleted)
- phases: probe→plan on a concluded probe_done (native: probe_done toolCall
WITH a clean result); plan→teach on the verdict marker (accepted:true).
- pending: `danglingQuestionOf` (+ input) is THE dock seam;
streamingToolActivity reads the live streaming message's open toolCalls.
- coverage: unit = native message (fork 3) with the call+result chunk rule;
messageViewLines goes native (same one-line vocabulary); the ledger
resets to the new units once.
- graph toolParts: native pairing (assistant toolCall + result lookup).
- extraction transcriptView: native lines.
- plan-blocks: derivePlanStatuses over the native transcript.
- GET /api/chats/[id] serves record.messages verbatim; page.tsx seeds the
client view model; store summaries read native scans.
## Files
New: client session module (hook + SSE parser + adoption), client view
model module (pure). Rewrite: ui-stream.ts (passthrough), coverage.ts,
phases.ts, pending.ts, graph.ts toolParts, extraction view, store/chats,
chat.tsx (mechanics transplant), ai-elements type swaps. Delete:
projection.ts, ui-message.ts, chat-transport.ts (retryingFetch moves into
the session module), `ai` + `@ai-sdk/react` from package.json. Routes:
unchanged. Agent band (src/agent): untouched.
## Test strategy
- Wire pin: scripted turn → SSE data lines are verbatim AgentEvents;
subscribe() replay = the same log; [DONE] terminal; turn-settled
unchanged; no suppression of question results (fork 2 pinned).
- View-model golden tests: the projection suite's expectations (grouping,
card attachment, splice, separators) become the view model's tests.
- Scanner natives: phases/pending/coverage (incl. the chunk rule)/graph/
plan-blocks.
- Hook tests: attach/replay/settle, mid-turn adoption, optimistic answer,
error banner, 409 identity retry.
- chat-flow rewrite: SSE assertions switch to AgentEvents; the answer path
drives the hook (closing the Wave B coverage gap that let the missing
stash ship).
- Live smoke: real turn + answer + reattach + remote abort on PORT=3004.
## Implementation order
1. Server passthrough (ui-stream + log type) — the wire changes here.
2. View model (pure) + scanner native rewrites + coverage chunk rule.
3. GET/page serve native; store summaries.
4. Session hook + chat.tsx transplant + renderer switch + ai-elements.
5. Deletes (projection, ui-message, chat-transport, deps) + gates.
6. Test migration + live smoke.
---
# Wave C planning — pi-native steer/follow-up delivery (2026-09-26 recon)
## Trigger
Live crash on first post-Wave-B steer delivery: a pump-fired wake turn
double-appended its own steer (the Wave B regression of the 2026-09-18
duplicate-key bug). Hotfixed as `c364013` (the carried filter restored;
record `chat_muhlcltz` deduped by hand). The hotfix makes steers WORK
again — Wave C is the structural migration onto pi's own delivery seams.
## What pi offers natively (verified in pi-agent-core source)
- `getSteeringMessages()` — injected at turn boundaries, mid-run.
- `getFollowUpMessages()` — polled when the agent would otherwise STOP;
returned messages join the context and the SAME run continues.
Emitted as full `message_start`/`message_end` events (our verbatim
wire carries them to clients for free) and pushed into the run's
`newMessages` (persistTurn persists them).
- `shouldStopAfterTurn` runs BEFORE both polls and EXITS the loop — the
existing `makeQuestionStopper` guarantees an open quiz is never
overridden by a queued steer. No new guard needed.
- `CustomAgentMessages` declaration merging — our `LhMarkerMessage`
(role "custom") can become a REGISTERED pi custom message type:
first-class in agent state, events, and newMessages.
`toLlmMessages` already is the sanctioned `convertToLlm` seam.
- `runAgentLoopContinue` takes no new messages: the wake's context must
already carry them. And pi polls follow-ups only AFTER the first LLM
call — an empty-input wake would produce a stray teacher turn before
the artifact arrives.
## Owner rulings (2026-09-26)
1. Hotfix first, Wave C after (done: `c364013`).
2. **Follow-ups ONLY** (owner corrected an initial steering choice):
turns are typically single-message; follow-up semantics reproduces
today's pacing (never interrupt a running turn) with least risk.
`getSteeringMessages` is NOT wired.
3. **One queue, no claim machinery** (the unified intent): the per-chat
steer queue is the single delivery registry; `claimSteers` and
continuation coalescing are DELETED. The idle wake carries its
markers as the run's input (write-ahead appends — today's semantics,
now the ONLY appender), because follow-up polls don't exist at run
start.
## The design
### Delivery paths (exactly one queue, three arrival contexts)
1. **Steer arrives mid-run** (teacher working): the agent's
`getFollowUpMessages` returns queued markers at the run's natural
stop; the same run continues (no second turn claim, no 409 dance, no
client re-attach — the marker streams over the live view).
2. **Steer queued during a quiz chain**: the answer's run stops on the
question stopper BEFORE the follow-up poll → the steer waits (exactly
today's waitForIdle semantics); it rides the first run that ends
without an open question — that run's follow-up poll delivers it.
(This replaces continuation coalescing: `claimSteers` dies.)
3. **No run at all** (client closed, idle): the pump keeps
`waitForIdle` + fires the wake turn carrying the queued markers as
its input appends (write-ahead durable, then the run responds).
The pump becomes: gates → fire ONE wake for everything queued → done.
The 409 backoff stays (a client turn may race the wake).
### Marker representation
- `declare module` augmentation registers `LhMarkerMessage` under
`CustomAgentMessages` — type-level only; the runtime shape is
unchanged (role "custom", customType, content, details, id,
timestamp). Markers stay the record shape (SubagentSteerCard,
probe-handoff, verdict rendering all keep working).
- `getFollowUpMessages` returns the marker as-is; pi pushes it into
context; `toLlmMessages` flattens it for the provider (unchanged).
### Idempotency + crash story
- Dequeue-at-poll: a follow-up marker leaves the queue when handed to
the loop; it reaches the record at that run's persistTurn. A crash
between poll and persist loses it — the SAME exposure today's
in-memory queue has (server restart re-derives steers only for
running jobs via job respawn). Not worse, not better.
- The wake path keeps write-ahead persistence (durable before the run).
- `steerDelivered` (id-in-transcript) remains the dedupe oracle at the
wake's append and for redelivery races.
### Client
- `applyWireEvent` extends to append NON-assistant messages arriving via
`message_start`/`message_end` (custom/user roles) — today it drops
them, and in-run follow-up markers arrive exactly that way on the
live stream. Assistant streaming keeps the slot logic (Wave D fix).
- Nothing else: `projectView` already renders steer markers; the wake is
still a turn (reattach paths unchanged).
### Scope boundaries
- **probe-handoff stays a wake input** (not a follow-up): the
prober→teacher switch is a record-level branch (`hasProbeDone` → the
next run builds the teacher); `prepareNextTurn` cannot swap a
toolset, so the handoff must start a new run regardless.
- Abort steers and pdf/research/demo/visual result steers ride the same
queue (kind is inert to the delivery mechanism).
- Extraction/coverage treatment of markers is unchanged (custom role,
view-line-less → invisible to coverage).
## Deletions
- `claimSteers` + its handler call site + the carried-filter complexity
(nothing left to double-append: the wake is the only appender).
- The pump's per-entry retry loop simplifies (one fire for the whole
queue snapshot; the 409 backoff survives).
- jobs.test coalescing suite → rewritten as follow-up delivery tests.
## Test plan
- jobs.test: queue → getFollowUpMessages handoff (dequeue-once);
quiz-open deferral (stopper exits before the poll); pump idle wake
fires ONE turn for N queued steers; 409 backoff unchanged.
- chat-flow: mid-run follow-up e2e — a scripted teacher run whose
follow-up poll returns a marker → the SAME stream continues with the
marker's message events + the teacher's response; record holds one
copy; `steerDelivered` redelivery no-op.
- session-core: custom-role message events append to the tail (wire
events from in-run injections).
- Live smoke on PORT=3004: dispatch a researcher job, deliver mid-turn
and idle, reload, abort steer.
## Implementation order
1. Declaration merging (type-level) + agentOpts wiring of
`getFollowUpMessages` reading the queue.
2. Client `applyWireEvent` non-assistant appends.
3. Pump simplification + `claimSteers` deletion + handler cleanup.
4. Test migration (jobs, chat-flow, session-core) + live smoke.
## Wave C pivot (owner correction, later 2026-09-26): follow-ups → STEERING
The owner re-examined the follow-ups-only ruling and overturned it: a real
learning session is ONE continuous run — teach a thing, verify it with a
quiz tool call, begin teaching the next thing. There are no "run would
otherwise stop" moments mid-lesson, and the few run ends that do exist are
exactly the ones where delivery must NOT happen (the question stopper). So
subagent results must be delivered MID-RUN, BETWEEN messages.
The pivot (all in the same two-vehicle skeleton):
- Vehicle 1 hands markers to `agent.steer()` — pi's STEERING queue. The
loop drains it at TURN BOUNDARIES: after a completed assistant message
AND its tool results (never between a toolCall and its toolResult — the
bonded pair survives injection), and again at RUN START (a continuation
drains the queue BEFORE its first LLM call). The run CONTINUES with the
marker in context.
- `steeringMode: "all"` (was `followUpMode`) — every handed marker joins
one boundary injection.
- `registerLiveAgent` moved BEFORE `runPiTurn` in the handler and now
hands pending markers itself: the loop's start poll drains the queue
SYNCHRONOUSLY inside the `runPiTurn` call, so a later registration would
miss it. A marker stranded by a quiz-terminated run therefore rides the
ANSWER run's first LLM call — the native replacement for the old
continuation coalescing.
- Settlement reconciliation, the idle wake, probe-handoff exclusion, and
the registry-as-truth design are UNCHANGED (stranding is now exactly the
stopper-exit case; the wake still covers a cold-and-idle queue).
Deferred after the pivot (owner's list): thinking blocks sometimes
rendering after messages; output tokens of certain tool calls vanishing
with no tool call actually happening.
docs/research/graph-rendering.md
-241
View File
@@ -1,241 +0,0 @@
# Graph rendering & cluster-first layout — research findings
*Compiled 2026-09-13 from four research passes (library fact-check ×2, prior-art ×2), all
claims source-verified by subagents against shipped package code (`npm pack` tarballs of
`sigma@3.0.3`, `@sigma/layer-webgl@3.0.0`), the sigma.js GitHub storybook source, and
primary docs. Speculation is marked where it appears. This document is the decision
basis for the rendering rebuild; nothing has been implemented yet.*
---
## 1. Requirements (as of this research)
Current symptoms to eliminate:
- Hull layers lag behind nodes during pan (desync).
- Settings changes shift hulls up-left until the next interaction (canvas-sizing race).
- GPU ~50–100% during pan/zoom (redundant full-canvas 2D repaints per camera event +
WebGL re-render per frame; the earlier `ctx.filter: blur` per frame caused coil whine).
- Stale/garbage boundary shapes at extreme zoom-out.
Feature set:
- Circles + directed arrows; precomputed layout (no runtime physics).
- Cluster regions: hull boundary around each community, dashed stroke, soft translucent
fill, stable color per region, region name label.
- Inverse zoom gates: node labels when zoomed IN, region labels when zoomed OUT,
cross-fading over a shared span.
- Pan, zoom-to-cursor, hover glow, click → detail card, edge-type filtering, camera
survives UI changes. 60fps, minimal GPU, ordinary laptop.
New direction (user, 2026-09-13):
- **Cluster-first placement**: layout depends on clustering — clusters placed as units,
nodes within a cluster close together (compact, natural hull shapes as a consequence).
- **Freeform experimentation**: a dev-only playground to try layouts/shapes/styles
against real and synthetic data; renderer must be dumb, data-driven, swappable.
Scale: 338 nodes / 211 edges today; low thousands expected (notes imports).
## 2. Verified facts about the current stack (sigma.js v3)
- **v3.0.3 ships a PUBLIC custom-layer API** — `createLayer` / `createCanvas` /
`createCanvasContext` / `killLayer` (typed in shipped `.d.ts`). Our current
`firstChild` DOM injection was improvising a worse version of a supported API.
- **Sigma sizes and DPR-scales ALL registered canvases** in its own `resize()` —
using the public API eliminates the canvas-sizing race by construction.
- **`afterRender` fires once per rendered frame** (rAF-debounced `scheduleRender()`).
Camera motion → full re-render each rAF. **Hover is cheap**: it only redraws the
`hoverNodes` WebGL layer + `edgeLabels` canvas via a separate rAF; no `afterRender`,
no full re-render. Sigma is strictly event-driven: no continuous loop, no rendering
when idle (verified: zero `setInterval`, all rAF sites are schedulers).
- **`renderer.graphToViewport()` is public and typed** — the sanctioned graph→pixel
path. `renderer.matrix` is **private**: building a `ctx.setTransform()` from it would
be guessing at internals; per-vertex `graphToViewport()` is the supported route.
- **The official cluster-label storybook example is exactly our pattern**:
custom DOM layer + `afterRender` + per-cluster `graphToViewport()` re-projection,
maintained in the sigma repo. Our minimal-change fix path is a demonstrated usage.
- Real v3 layer order (verified in source, NOT in docs):
`edges` (WebGL) → `edgeLabels` → `nodes` (WebGL) → `labels` → `hovers` →
`hoverNodes` (WebGL) → `mouse`.
### `@sigma/layer-webgl@3.0.0` (official background-layer package)
- `bindWebGLLayer` inserts a background WebGL layer (`beforeLayer: "edges"`), redrawn
on `afterRender` with the same camera matrix — fully in-loop, no desync.
- **`createContoursProgram` renders METABALL-style implicit fields, not hull polygons**
(verified from the fragment shader: fullscreen quad, summed radial falloffs,
analytic contour lines via `dFdx/dFdy`). Translucent fill: yes (arbitrary CSS color
with alpha). **Dashed stroke: NO** — solid anti-aliased line only; no dash parameter
exists. Per-community color: yes, but one program + one WebGL context per community,
and each runs a fullscreen fragment pass per frame (browsers cap ~16 WebGL contexts —
a problem for dozens of regions).
- **Decisive fact**: our exact visual (hull polygon + translucent fill + dashed
stroke) is **NOT achievable with shipped layer-webgl programs**. It would require a
custom `WebGLLayerProgram` (supported extension point, moderate effort, dashes DIY in
GLSL) — whereas Canvas 2D gives it for free with `ctx.setLineDash()`.
- v4-era programs (heatmap, density-splat, dashed/fill layer factories) are NOT in the
v3 package.
### sigma v4
- Real but **beta** (4.0.0-beta.5; docs site self-describes as alpha). Single `stage`
canvas + public `createLayer`, GPU picking with downsizing, SDF labels, depth
layers. Breaking migration. Not a production bet today.
### Pan GPU cost at our scale
- Structural facts verified (event-driven idle; full re-render per camera frame;
synchronous `gl.readPixels` picking per mousemove pre-#1546, since fixed).
- **Absolute GPU cost of panning 338–3k nodes in v3: UNVERIFIED** — no public
benchmark exists; nobody has measured it. If it matters, we measure it ourselves.
## 3. Verified facts about the candidates
| Candidate | Renderer | True hull fill+dash | Picking | Camera | Maintenance | License | Fit |
|---|---|---|---|---|---|---|---|
| sigma v3 + public 2D canvas layer | WebGL (+our 2D) | yes (our canvas, free) | GPU color-pick | built-in | active (3.0.3, 2026-04) | MIT | 7/10 |
| sigma v3 + layer-webgl contours | WebGL | **no** (metaballs, solid border) | — | built-in | active | MIT | 6/10 |
| sigma v4 + layers | WebGL, SDF | via custom program (beta risk) | GPU picking | built-in | **beta** | MIT | 6 (risky) |
| d3-zoom + Canvas 2D (hand-rolled loop) | Canvas 2D | yes, trivially | distance check | d3-zoom (verified math) | d3 very active | ISC | **8/10** |
| PixiJS v8 + pixi-viewport | WebGL/WebGPU/Canvas | yes (Graphics; dashes via shim `pixi-dashed-line`) | built-in events | pixi-viewport 6.0.3 | very active | MIT | 7/10 |
| cytoscape.js 3.34 | Canvas 2D | via bubblesets/layers extensions | built-in | built-in | active | MIT | 5/10 |
| vis-network 10.1 | Canvas 2D | **no** — clustering = supernodes (collapse, opposite of our regions) | built-in | built-in | sporadic | MIT | 2/10 |
| @cosmos.gl/graph (Cosmograph engine) | WebGL (luma.gl) | no (cluster = layout force only) | GPU | built-in | active | MIT | 3/10 |
Fact nuances we must not over-trust:
- "Canvas 2D is GPU-rasterized": **half-true**. Skia may raster on CPU or GPU depending
on browser/OS config. Either way the cost driver is **repaint count × pixel area
(DPR²)**, not "Skia = GPU".
- Canvas 2D at 60fps for hundreds–low-thousands of elements: **supported by practitioner
evidence** (cytoscape is fully canvas-based into the thousands; 10k animated canvas
elements ≈ 55fps in one 2025/26 benchmark). Pixel area matters more than node count
at our scale.
- Obsidian's graph view renderer: **UNVERIFIED** — no primary source. Do not cite.
- PixiJS exact bundle size: unmeasured. yFiles: commercial, unverified details.
## 4. Prior art for our specific features
### Region/hull visuals
- Nobody in the graph-tool world ships our region aesthetic: Gephi desktop, Gephi Lite
(the flagship sigma app — zero hull code in its tree), Obsidian — all partition by
**node coloring only**. The region feature is a differentiator, not a copied pattern.
- The aesthetic reference: ggforce `geom_mark_hull` (expand + round the hull, low-alpha
fill, integrated label placement).
- **Bostock's thick-stroke hull trick**: one path, wide stroke (e.g. 32px) +
`linejoin: round` = padded, rounded hull with zero polygon-offset math. Replaces our
offset-normal code outright.
- Named-region UX validation: Excalidraw frames / tldraw regions (dashed titled
container around content).
- Closest feature twin: **Cosmograph** — documents "cluster labels when zoomed out,
point labels when zoomed in" as a first-class use case (our exact inverse gate).
### Zoom-gated labels
- Consensus pattern = map-style per-layer zoom ranges (Mapbox/MapLibre
`minzoom`/`maxzoom` on symbol layers). State labels at far zoom, city labels at near
zoom is the standard cartographic solution; our inverse gates are the same mechanism
with opposite ranges. Sigma's own `labelRenderedSizeThreshold` is the zoom-in gate.
- Cross-fade + hysteresis over a shared span: no single canonical writeup (medium
confidence); nearest established practice is map label collision systems.
### The sync pattern (fixes our desync class structurally)
- Canonical and repeatedly documented: **one rAF loop with a dirty flag; input/DOM
events only mutate state.** All layers drawn in the same frame, under the same
camera transform — either same canvas (`setTransform` once, draw back-to-front) or
stacked canvases all redrawn by the same loop.
Sources: browser.engineering, MDN "Optimizing canvas", web.dev canvas performance.
### Picking
- Canvas 2D at our scale: nearest-node distance check per pointer event is the norm,
trivially fast (µs for 3k nodes; spatial grid only past ~10k).
## 5. Cluster-first layout (the new direction)
- **Two-level pattern is well-trodden**: place cluster anchors first (quotient graph —
one super-node per community, Gephi's "Groups by Partition" plugin is the desktop
prior art), layout nodes within each cluster, compose by translating local layouts
to cluster anchors. Bostock's Clustered Force Layout lineage; cosmos.gl's cluster
force verified from source (per-frame center-of-mass pull, or user-pinned positions —
cluster anchors as data independent of member layout).
- **graphology FA2 has NO cluster-aware settings** (full settings list read; gravity is
global). **LinLog mode** (`linLogMode: true`) is the documented "communities separate
tightly" switch — a one-afternoon A/B candidate before any custom pipeline.
- **No turnkey JS package does the composed two-level pipeline** — assemble from
graphology primitives (~150 lines of glue: quotient build, per-cluster FA2 with
circular init, compose transform, `noverlap` cleanup). The combination is our
synthesis from verified primitives.
- Deterministic given fixed init positions (no RNG dependency in FA2).
- **Server-side at extraction time is the better fit** for our architecture: positions
become data, rendering stays dumb. Browser worker fallback for live re-layout.
- Harness precedents: yFiles layout playground (polished), Observable/bl.ocks lineage
(notebook = graph + sliders + canvas), react-sigma layouts example. No canonical OSS
playground for our stack — copy the config-driven pattern (algorithm dropdown +
parameter sliders + seed input + live canvas).
## 6. Options (the decision)
**Option A — Stay on sigma v3; mount the hull canvas via the public
`createCanvas` API; redraw on `afterRender` via `graphToViewport()`.**
Supported, minimal-change, demonstrated by sigma's own cluster-label storybook. Kills
the desync, the sizing race, and the multi-event redraws (once per frame, sigma-sized).
Our exact visuals stay (dashes free in 2D). Keeps sigma's camera/labels/picking
machinery. Residual: full-canvas 2D repaint per interaction frame remains (raster cost,
mitigable with move-gating like sigma's own `hideLabelsOnMove`); WebGL + 2D raster
still both run per frame. Est: ~1 day.
**Option A′ — sigma + custom `WebGLLayerProgram` for hulls.** Supported extension
point, but dashes are DIY GLSL and we'd rebuild in shaders what Canvas 2D gives free.
No reason given A exists. Est: several days, for a worse iteration loop.
**Option B — d3-zoom + hand-rolled single Canvas 2D renderer.**
One canvas, one rAF + dirty flag, world-space geometry + one transform, all layers
back-to-front in the same frame. Lowest GPU/CPU, total control, perfectly matches the
new requirements: renderer is a dumb draw function over data (layout experiments can't
fight it), playground is trivial, cluster-first layout slots in as pure data. Costs
~600 lines + camera feel + picking + label gating owned by us. Est: ~1 focused day.
**Option C — PixiJS v8 + pixi-viewport.** Cleanest scene-graph answer (one ticker,
sync by construction), WebGPU headroom. But: dashes need a third-party shim, no graph
semantics, bundle, more machinery than Canvas 2D needs at our scale. Est: 1–2 days.
**Recommendation**: **Option B as the target** — it is the only option that structurally
matches all three requirement sets (bugs, cluster-first placement, freeform
experimentation), scored highest (8/10), and removes the vendor-internals class of
problems permanently. **Option A is a legitimate ~1-day stabilization** if we want the
current bugs dead immediately while deferring the rebuild; it reuses all our existing
visual code and is the officially sanctioned pattern, but it keeps us on sigma's
opinionated machinery and the experimentation direction will keep pushing against it.
**Layout work rides along independent of renderer choice**: first A/B
LinLog-only FA2 (one afternoon), then the two-level pipeline
(quotient → per-cluster FA2 → compose → noverlap) if needed, developed in the playground.
## 7. Facts we should NOT rely on
- "Obsidian's graph view uses renderer X" — unverified.
- "Sigma's documented canvas DOM order" — source-only, implementation detail, changed
in v4.
- "Canvas 2D is always GPU-rasterized" — false as stated.
- "Sigma v4 is ready" — beta.
- "Sigma pan cost at our scale is X%" — nobody has measured it publicly.
- "@cosmograph/cosmos is the maintained Cosmograph engine" — stale npm channel (Dec
2024); current engine is `@cosmos.gl/graph` v3.
- Any PixiJS bundle-size number — unmeasured.
## 8. Open items for the implementation phase
- Measure pan GPU/CPU ourselves (Chrome profiler) before/after whichever path is chosen.
- Playground design: `/playground` dev-only route, config-driven (layout algorithm
dropdown, parameter sliders, seed input, synthetic + real datasets), same renderer.
- Hull styling decision inside Option B: Bostock thick-stroke trick vs offset polygons
(geometry already unit-tested in `components/cluster-geometry.ts`).
- Label-gate hysteresis: small dead-band in log-zoom to avoid threshold flicker.
- Whether layout runs server-side at extraction (persisted as data) or in-browser.
## Center-anchor on window resize (2026-09-17)
The camera translate is measured from the canvas's TOP-LEFT corner, so a
window resize used to keep the top-left pinned and the graph appeared to
slide by half the size delta (owner report). The resize observer now
captures the world point under the PRE-resize viewport center (from the
last-synced `viewW/H` — the ResizeObserver already sees the new size in
clientWidth/Height), then re-anchors the translate so that same world
point lands under the NEW center (applied through the zoom behavior, so
d3 state, camera persistence, and card-dismiss all behave like any
camera move). Zoom (k) is untouched; `refreshFitZoom` metadata is
viewport-bound and recomputed as before. Verified by canvas pixel
sampling: 7/7 center-area samples identical across a 300×200 shrink.
docs/research/stt.md
-74
View File
@@ -1,74 +0,0 @@
# STT research — verified findings (2026-09-17)
Two researcher passes over current (Sept 2026) sources, cross-checked
against our codebase. Decisions at the bottom.
## Cloud: Deepgram (the default)
**Verified facts**
- Streaming is a **WebSocket API** (`wss://api.deepgram.com/v1/listen`); REST is batch-only. Query params carry config; binary audio frames up, JSON `Results` down; client messages `KeepAlive` / `Finalize` / `CloseStream`.
- JS SDK: `@deepgram/sdk` **v5** (v5.11.0, Sept 2026) — browser-capable WebSocket (live transcription works in the renderer directly; only REST needs a CORS proxy). `listen.v1` socket = Nova; `listen.v2` = Flux (conversational, turn-taking — overkill for dictation, skip).
- Model: **`nova-3`** (still the flagship general/dictation model as of Sept 2026; no Nova-4). `language=multi` exists for code-switching.
- **Auth: API key via `Authorization: token` header.** For client-side use, Deepgram recommends **short-lived access tokens** (`auth.v1.tokens.grant()`) instead of embedding the key — fits our architecture exactly: key stays in auth.json → server env; `/api/stt/token` mints a token; renderer opens `wss` directly with it. No audio proxying through our server, no `ws` dependency on the server.
- Pricing: streaming Nova-3 **$0.0048/min promotional** (regular $0.0077), pre-recorded $0.0043/min; per-second billing. **$200 one-time free credit**, no card. Diarization +$0.0020/min streaming — skip (single-user dictation).
- `http://127.0.0.1` is a **secure context** (potentially trustworthy origin per the Secure Contexts spec) — `getUserMedia` works without TLS. Our permission handlers (request + check, media-only, loopback-origin-only) are already the documented required pattern.
- macOS: `NSMicrophoneUsageDescription` already in the packaged build. TCC bites fully only post-signing — test the packaged signed app later. Call `app.focus()` / `win.show()` before first capture so the TCC prompt isn't behind the window.
**Capture: AudioWorklet → linear16, 16 kHz mono** (not MediaRecorder)
- `getUserMedia({ audio: { sampleRate: 16000, channelCount: 1, echoCancellation: true } })`, Float32→Int16 in an **AudioWorklet** (off-main-thread), stream linear16 frames with `encoding=linear16&sample_rate=16000&channels=1`. Deterministic format, no container surprises (webm/opus streaming has unresolved community reports of empty transcripts), no encoder lookahead, ~100–200ms chunks. This is what Deepgram's own browser layer does. The worklet module ships as a static file under `public/` — rides the standalone bundle for free.
- Zero new npm deps for the cloud path: getUserMedia + Worklet + WebSocket are platform APIs.
**Gotchas that shape the design**
- **10s idle timeout** (server closes with NET-0001): send `KeepAlive` text frames every 3–5s during silence. Timestamps resume contiguously — long pauses in a dictation run are safe.
- **Interim results**: `interim_results=true` → partials (`is_final=false`) to render live; finals (`speech_final=true` / `is_final=true`) commit. `speech_final` (endpointing-driven) and `is_final` (model-driven) are different flags — check both.
- **Endpointing + UtteranceEnd**: endpointing uses Deepgram's server-side VAD (`endpointing=500` for a real pause threshold; the documented "10 ms default" is a doc quirk — always set it explicitly). Add `utterance_end_ms=1000` as the noise-robust backstop (word-timing-gap based, needs `interim_results=true`).
- `smart_format=true` (punctuation, caps, numbers/dates) — but it can *delay* endpointing while formatting e.g. phone numbers; keep UtteranceEnd on.
- **No server-side buffering on disconnect**: reconnect + resume is our job (SDK v5 auto-reconnect, cancellable via `abortSignal`).
**Alternatives (why still Deepgram)**: OpenAI live-transcribe $0.017/min (~3.5×), no word timestamps/confidence, 60-min session cap. AssemblyAI Universal-Streaming $0.0025/min is cheaper but smaller ecosystem. Speechmatics best for non-English but ~2×. GCP is service-account-shaped — bad BYOK fit. Verdict: Deepgram stays the pragmatic default.
## Cloud: Deepgram (the default) — landed
**Live-verified facts (2026-09-17, cloud wave)**
- The **grant response field is `access_token`**, not `token` — `POST /v1/auth/grant` returns `{ "access_token": "<jwt>" }`.
- **Minted JWTs authenticate over the WebSocket via the `bearer` subprotocol**: `new WebSocket(url, ["bearer", jwt])`. The `token` subprotocol is for **raw API keys only** (the pi dictate extension's path) — `["token", jwt]` is rejected at the handshake (TLS-level close, no 401).
- **The key needs ≥ Member permission** (console → API Keys → Create Key → *Advanced* options → permissions dropdown → Member). A default key can transcribe but gets `FORBIDDEN` from `/auth/grant`.
- **Env precedence gotcha**: a real `DEEPGRAM_API_KEY` exported in the shell (e.g. `.zshrc`) **beats `.env.local`** in both `next dev` and the desktop shell's `authEnv()` seam. Stale exports silently override the file — don't export the key in shell rc files; `.env.local`/auth.json are the canonical seams.
## Local: whisper.cpp (the Advanced option)
**Verified facts**
- **Ship the whisper.cpp CLI (v1.9.4, MIT) as a child process** — the 2026 community norm; every maintained Node wrapper (nodejs-whisper, whisper-node) is just a CLI spawner anyway, and native bindings (smart-whisper last published Oct 2024; the official `addon.node` is a self-compiled reference demo) add electron-rebuild/ABI churn for nothing. Our server already spawns children; per-utterance spawn is fine (model load <1s, stateless).
- **Metal-accelerated** on Apple Silicon: RTF on M-series with Metal — tiny 0.02, base 0.04, small 0.08 (every Apple Silicon Mac runs `small` faster than real time; ~4.4× slower on CPU-only).
- Models (ggml, MIT): **base.en 142 MB** (WER ~5.5%), **small.en 466 MB** (~3.4%) — our default + quality tier. Skip medium (1.5 GB / 2.5 GB RAM). RAM ≈ 3–5× file size. Cache under `<LH_DATA_DIR>/models/` (our existing model-cache seam), download on first use, checksum-verify; distil variants need self-hosting — skip for v1.
- **transformers.js Whisper is a non-goal for v1**: no Metal-verified Node path, no credible Apple-Silicon ONNX benchmarks, weaker ASR catalog. (It was originally kept as the "zero new native binary" fallback because transformers.js + onnxruntime-node already shipped — that premise died with the wave-7 uninstall, 2026-09-24: neither is in the payload anymore, so the fallback would now be *new* payload.)
- Apple Speech (SFSpeechRecognizer/SpeechAnalyzer): 1-min cap, 1k req/h, second permission prompt, macOS-26 gating for the good path, WER ~5.5% vs whisper.cpp small 3.4% — defer.
**Local flow (per-utterance batch, honest trade-off vs cloud streaming)**: energy VAD in the worklet → ~1s silence closes the utterance → POST the buffered PCM to the server → 44-byte WAV header, no ffmpeg → spawn `whisper-cli --model <path>` → text appended to the composer.
## Decisions (owner, 2026-09-17)
- **Activation: toggle mic** — click to start a dictation run, click again (or Esc) to end; no push-to-talk before the shortcuts milestone.
- **Wave scope: cloud first, local second** — capture component + Deepgram path ship and get live-smoked as one wave; whisper.cpp follows as a second wave.
- Renderer never holds the real key (server mints short-lived Deepgram tokens); audio never loops through our server (renderer → wss direct).
- Engine picker = small popover off the mic button (localStorage for now; settings milestone absorbs it).
- Bands: 1) capture + mic states (no engine), 2) cloud end-to-end, 3) local end-to-end, 4) smoke + docs.
## Round-1 refinements (owner, 2026-09-17 — first live mic test)
- **No interims.** Live words beside the meter are distracting AND shift the meter around — the meter's width stays fixed while recording. Params now match the pi dictate extension exactly: `interim_results` omitted (false), `endpointing=300`, no `utterance_end_ms`.
- **One primary transcription input at all times**: while a quiz/ask question waits, finals land in ITS field (text mode → "Your answer", select modes → "Your thinking") — never the hidden composer. Implemented as a module-level claim registry (`useDictationTarget`); the composer is the fallback when nothing claims.
- **Optimistic meter**: the click enters the meter view immediately (Stop pill + meter within one frame); the token mint and mic acquisition run concurrently (Promise.all) behind it. Esc or click-again stops.
- **Wide idle button**: the stats-bar middle is one full-width affordance — a quiet bordered "Dictate" button when idle (error text replaces the label on failure), the Stop pill + fixed meter while live.
- Final-appending keeps the dictate-flush separator rule (`joinDictation`).
## Round-2 refinements (owner, 2026-09-17 — second live test)
- **Deferred transcript**: finals buffer during the run and flush into the primary input ONCE at the end (socket-close → one joined append). Nothing textual renders while recording. Live output may return as a toggleable setting at the settings milestone; deferred is the default.
- **Adaptive meter**: cell count derives from the meter's measured width (4px pitch, clamped 8–48, ResizeObserver; cells have no minimum width). Draft/home rows beside the knowledge-context slider show ~20 cells; chat shows the full 48. No more overflowing under neighbors.
docs/vision-assessment.md
-81
View File
@@ -1,81 +0,0 @@
# Vision — Assessment Levels (Mastery)
Started 2026-10-06, thinking round (owner + assistant). **Deliberately unscheduled**: first release (0.3.0) comes first; this is the post-release arc. It resolves the "aggregation TBD" left open in vision-practice.md's assessment-record model (2026-09-27 rulings) — and inherits that round's law: *understanding is inferred, never encoded*.
The surface for all of this is the future **data view** (still undesigned), precisely because these are computed estimates, not truths. Nothing here writes to the graph (the 2026-09-28 cleansing ruling stands: propositions + edges only, no understanding fields).
## The unit: propositions (ruled 2026-10-06)
Mastery is assessed **per proposition**. Not per chain, not per session, not per learner-skill.
- The infrastructure already speaks propositions: practice problems tag target proposition ids, validated whole-graph at pose time (the round-9 mastery-trace contract, ids-first).
- The worry "knows all propositions ≠ knows the chain" is real but is an **instrument concern, not a model concern**: composition-mode problems *are* chain assessments (you cannot pass one without executing the inference), and their evidence already lands on member propositions. A chain whose propositions are all green but which never appears in a composition item is a problem-generation coverage gap.
- **Chain readiness is derived on read, never stored**: min (or product) over premise+conclusion mastery. For selection and display only. No second ledger.
## What the number means: forecast semantics (ruled 2026-10-06)
> **Mastery(p) = our current estimate of the probability that the learner can correctly exercise proposition p in a fresh, unaided retrieval.**
We do not claim to observe understanding; we forecast performance. This makes the number honest (it is a probability, presented with its uncertainty, in the data view — never a verdict), naturally 0..1, and directly consumable: the doctrine's ~85%-success-per-retrieval target becomes selection arithmetic (pick propositions whose forecast sits in the ~0.85 band, decaying ones, or agent-flagged `retest`).
It also gives the parked Brier idea its precise home: each new observation *scores* the pre-update forecast — (forecast − outcome)². "Are our numbers honest?" becomes measurable; a calibration panel is a natural data-view neighbor.
Terminology: **mastery** is the display word. The precise phrase is *estimated recall reliability*; the definition carries the precision.
## The estimator: a Beta belief with decay (agreed direction 2026-10-06)
The owner's two candidates — BKT per proposition, or a weighted average of outcomes — resolve as: **BKT is the wrong tool; the weighted average is the right skeleton, completed with a prior, time decay, and uncertainty.** That completion makes it a Bayesian model anyway.
Why BKT fails here:
- **Binary observations.** Our items are graded by the agent's rubric (outcome × rung × missClass — the richest signal we have); binarizing throws most of it away.
- **The guess parameter is fictitious** — the model sees the learner's thinking. Slip survives only as careless/execution misses, which missClass already *names* rather than smooths as noise.
- **Identifiability.** BKT wants p-init/p-learn/p-slip per skill; we will have ~1–10 observations per proposition. Per-proposition parameters cannot be fit; global parameters degenerate BKT into an EMA with ceremony.
- **BKT never forgets** (p-known is monotone). Mastery must decay with disuse.
Three layers:
**1 — the evidence scalar.** Each *concluded* problem becomes one number e ∈ [0,1]: "fraction of unaided mastery demonstrated." The agent's rubric, numerized — a tunable table, not physics:
| outcome | e (approx.) |
|---|---|
| solved, no hint | ~0.95 (never 1.0 — one clean solve is not certainty) |
| solvedWithHints | ~0.85 → ~0.3 as rung climbs R1→R5 |
| movedOn | ~0.25 |
| surrendered | ~0.1 |
| dismissed | **excluded entirely** — nothing was demonstrated, in either direction |
missClass modulates within a cell: *careless*/*execution* misses are failures not about the proposition (discount the negative evidence); *cue* and *structural* are the real negative signal; *discrimination* sits between.
**2 — the belief.** Per proposition a Beta(α, β); each observation adds fractional success/failure counts weighted by:
- **target share 1/k** — uniform distribution across the problem's targets (correct, not just convenient: a k-target composition item genuinely spreads its evidence; counting it k full observations would inflate confidence k×);
- **recency** — exp(−Δt/τ), τ in weeks (forgetting; old evidence fades toward the prior);
- later, if the perimeter ever widens: an instrument weight.
Posterior mean α/(α+β) is the displayed mastery; the spread is the honest error band (`0.62 ± wide · ~3 effective observations · last practiced 6d ago` — two data points visibly ≠ twenty). New propositions start at a weak prior (order Beta(2,5) ≈ 0.29 — propositions come from the learner's own material; low concentration so 2–3 observations move it decisively). Every knob (the e-table, τ, the prior) is a number in one place, tunable from field data without touching records or prompts.
**3 — derived, never stored.** Chain readiness from member mastery (above).
## Credit assignment (default ruled; the upgrade open)
Default: uniform 1/k. The finer detail the owner wanted has a cheap shape, **unruled**: an optional `missed: [propositionIds]` field on the conclude call — the agent names which target(s) the miss was about, validated against the pose's own targets (round-9 discipline). Negative evidence then concentrates where it belongs; the other targets take fuller positive credit; "one third of a failure" becomes "the failure has an address." Rejected: per-submission grading (the conclude is the graded unit; grading every thinking message over-weights long problems and doubles agent work) and string-mining the verdict prose (drift).
## Outcome levels: no new enum values (ruled 2026-10-06)
The five-value outcome enum stays. outcome × rung × missClass already spans the space; felt missing resolution ("solved with a nudge" vs "solved after a walk-through") is carried by the rung. Resolution belongs in the scoring table (tunable in an afternoon); enum growth taxes every future conclude call — weak models, weak schemas (the GLM incident's lesson).
## The perimeter: practice sessions only (ruled 2026-10-06)
**Practice sessions are the sole source of assessment data.** Teaching sessions never contribute, even though they contain proposition-shaped evidence (quizzes, explain free-recall, predict/check). Reasons: clear boundaries and a single truth per construct; teaching is an entirely different purpose (active, just-taught material vs cold retrieval); and concretely, the nodes quiz questions reference are session-scoped plan objects, *not* graph propositions — no honest mapping exists (the 2026-09-27 guardrail). The evidence record still carries a `source` field internally, but only practice writes it.
## Storage: nothing new, on principle (ruled 2026-10-06)
The raw evidence is already durable: problem calls carry targets + mode; conclude calls carry outcome + rung + missClass; both live in append-only chat records under honest activation. **The assessment trace is a join over what exists; mastery is a computed view** (like graphSnapshot) — cached only if it ever gets slow. If the practice agent later needs mastery as a selection input, it reads the computed view at session start. No stored ledger, no drift with the records, graph untouched.
## Open questions (unruled)
- The prior's center: Beta(2,5)-ish, or higher for propositions from the learner's own documents?
- The `missed` field on conclude: adopt or not?
- τ (the decay half-life): weeks — set from field data once it exists.
- The e-table's exact cells: calibrate after the first real corpus of concluded problems.
- Whether mastery ever feeds the practice agent's *live* problem selection during a session (the arc's second half — adaptive selection) or only at session start.
- The data view's presentation of all of this (its own design round).
docs/vision-branching.md
-23
View File
@@ -1,23 +0,0 @@
# Vision — Session Branching
Started 2026-09-27, ideation round. No priority assigned yet (suggested slot: after practice + side chats — it is the deepest record-model change in this document set).
Now that the app is fully pi-native, support **session branching**: the message history becomes a tree; the learner can navigate back in time and branch — explore "what if" continuations, revisit decision points, keep alternate lines.
## The UI (owner sketched 2026-09-27)
A popup overlay **in the chat area** — exactly the same overlay pattern as the plan window. It presents the session's **teacher messages, separated by teacher message**, and the user branches from those points.
## Assessment (coordinator, 2026-09-27)
- The overlay UI is the easy part. The record today is a **flat native message array**, and everything downstream assumes linearity: the view projection/grouping canon, the write-ahead persistence, the turn runner's one-turn-per-chat liveness, coverage fingerprints (keyed by message id), extraction idempotency (refs by toolCallId), subscription replay. Branching means a **tree-shaped record model** designed deliberately — not a UI feature bolted on.
- **The pi SDK carries no native session-tree branching** (its `Branch*` exports are compaction summarization only, verified 2026-09-27). This is our own record-model work. Per the owner: read the full pi docs on sessions/branching before designing — confirmed as implementation-time step 1 regardless.
- Value is real: rewind/replay fits the tutoring domain well (alternate explanations, "what if I'd answered differently"), and it composes cleanly with the parked persistent-agent idea later (branches as parallel histories to reconcile).
## Open questions (unruled)
- Tree representation: parent pointers per message vs. branch records referencing a base prefix.
- What happens to graph evidence from **abandoned branches** — does extraction run per-branch, active-path only, or all paths with branch-scoped attribution?
- Branch lifecycle: naming/labels, switching, deletion; is the original line just another branch?
- Are user messages also branch points (edit-and-resend), or teacher-message anchors only, per the sketch?
- Interaction with practice/side chats (presumably none — they attach to their own surfaces).
docs/vision-context-injection.md
-54
View File
@@ -1,54 +0,0 @@
# Vision — Graph context injection
Ideation settled 2026-09-14. Not yet planned, not yet assigned to a wave. This document is the decision record; `vision.md` § "The graph as session context" settles its open *selection problem* here (V1 only).
One line: a new session's first message is its **topic**; the topic is embedded, the semantically closest propositions from the global knowledge graph are selected under a token budget, and delivered to the session as a **steer** — *"what this learner already knows."* This is the embeddings investment paying off: the same vectors that power clustering now personalize teaching.
---
## The mechanism, end to end
**Trigger — one-shot, at session start.** The injection job fires when the session's first user message exists (the topic). One shot per chat, idempotent, deterministic steer id — same discipline as every other job/steer. No mid-session re-injection (parked), no retroactive injection for sessions already in flight.
**The topic is a query; the propositions are documents (DECIDED 2026-09-14).** EmbeddingGemma's model card prescribes *role-asymmetric* input formats: retrieval queries use `task: search result | query: <text>`, retrieval documents use `title: none | text: <text>`. The ledger already embeds propositions in the document format — it stays untouched. The topic embeds through the **same warm embedder singleton** with the **query format**. The requirement is not "same format" — it is **same model, same 512-dim MRL truncation** (one vector space), with each side in the format the model was trained for. The job asserts the ledger's model+dim stamp before scanning; there is never a parallel embedding path.
- *Why formats matter at all:* semantics dominate — raw text would still roughly work — but the benchmark quality this model was picked for is measured with the trained templates, and using them costs nothing. The embedder's own header already warns that mixing prefixed and unprefixed inputs degrades quality.
- *Rejected:* re-embedding the whole corpus as plain `title\ntext`. It voids every vector (a minutes-long full re-embed) for a format the model wasn't trained on. If retrieval quality ever looks weak on real data, the graph stream can benchmark plain-vs-card formats against the 2k-node harness — an experiment, not the default.
**Search — brute-force cosine, exact, no index.** Scan the in-memory embeddings ledger for top-K similarity. ~1M multiply-adds at 2k propositions (sub-millisecond); ~25ms at 50k. No ANN, no vector DB — not until ~10⁶ scale, which is far away.
**Latency is a non-issue.** The topic embed takes tens to ~200 ms warm (extraction keeps the pipeline singleton hot); the whole job — embed, scan, build block, queue steer — is comfortably under a second warm. Cold start is seconds (weights load once), and harmless because delivery is async anyway.
**Selection — pure token budget; similarity only ranks (DECIDED 2026-09-14).** The user-facing control is a **token budget** — there is deliberately **no similarity threshold**. Irrelevant-but-cheap content rarely hurts; the only real hazard is *too much context*, and token count is what governs behavior. Fill greedily in similarity-descending order, stop before exceeding the budget, plus a hard count cap (tuned at build time) as belt-and-braces.
**Token estimate: chars / 4.** Heuristic, no server-side tokenizer — good enough for a budget dial and for the observability card.
**Annotation — UnderstandingLevel, folded most-recent (DECIDED 2026-09-14).** Levels live **per extraction instance**, tied to the moments a proposition appeared in sessions; a node can carry several across its life. The injected block reports the level from the **most recent instance** that touched the node: latest evidence is the best estimate of current understanding, because learners progress — an optimistic fold (highest ever) risks planning on stale claims, a pessimistic fold (lowest) wastes ground actually covered. Recency also keeps `resisted` visible exactly when it is current. The block's gloss: **solid** = build on it, **shaky** = target it, **resisted** = teach around it, **not yet engaged** = no evidence yet, **accepted** = took it in.
**Delivery — a steer through the existing wake pipeline, gate-for-gate identical to sub-agent results.** The prober typically opens with a run of quiz/ask calls; the steer queues behind the pending question and rides the turn after the learner answers. The agent never "waits up" — delivery lands at the natural seam mid-probing, well before plan_init, which is where prior knowledge matters most. Cost: one visible prober turn acknowledging the context. Accepted as a *feature*: the learner watches the teacher recognize what they already know — the product's premise, enacted. The transcript keeps the audit trail: exactly what was injected, and when.
**V1 keeps it raw (DECIDED 2026-09-14).** Plain similarity-ordered top-K. No dedup or cluster-representative substitution, no `follows`-links among the injected set, no frontier/diagnostic ranking — see how raw retrieval works and feels first. All of those stay parked.
---
## The setting
One field on **ChatSettings** (rides the same pre-chat `PUT /api/chats/[id]/settings` as model + thinking, so the draft view gets it nearly free): the graph-context token budget.
**Default 1024, range 0–4096, step 128, 0 = off (DECIDED 2026-09-14).**
**Placement (DECIDED): the blank session-data section of the graph view, for unstarted sessions.** The symmetry is the point: the panel that will later show what the session contributed to the graph starts by asking what the graph will contribute to the session — input before output.
**The topic reframe is presentational, not structural.** Message 1 is functionally already the topic (it is the title, the opener, the embedding source). Parked refinements: the draft composer framed as a topic ask (matching home's "What do you want to learn?"), and message 1 rendered as a quiet topic header rather than a chat bubble. The record contract is unchanged — zero adapter churn; the embedder reads message 1's text.
---
## Observability
- The job appears in the **background-processes dock** (a new JobKind in the existing pattern) from send-of-first-message to steer delivery.
- The delivered steer renders as a **quiet marker card in the transcript** carrying the numbers: token estimate and proposition count (e.g. *"knowledge context — ~1.1k tokens · 23 propositions"*). Same marker-part pattern as verdicts and handoffs; the model-facing block text is adapter/canonical-owned like all steer text.
- A small prompt line (agents lane, at build time) tells prober/teacher to treat the block as reliable prior knowledge.
## Lane split — decided at planning time, not now
Graph owns the embedder service + ledger scan; agents own the JobKind, the steer, and the prompt line; UI owns the slider; master owns the ChatSettings contract and the seam. Nothing here is assigned or scheduled.
docs/vision-edge-glyph.md
-46
View File
@@ -1,46 +0,0 @@
# Vision — The edge glyph: follows vs supports
Ideation settled 2026-09-14. **WITHDRAWN 2026-09-15**: both shipped variants of
the double-shaft glyph (tails converging on a shared head; two complete arrows
offset ±half-gap) were rejected by the owner live — the first "two arrows
feeding one point", the second "two arrows next to each other, not coherent".
Dashed was already rejected 2026-09-14 (dash patterns crawl under the constant
screen-space stroke width). **The relation still needs a differentiator —
approach open.** The constraints below that are NOT glyph-specific still stand.
Companion: `vision.md` § "Edges" (the relation types) and § "Rendering".
One line (withdrawn): the two relation types become visually distinct —
**`follows` renders as a double-shaft arrow** (two parallel tails side by
side, the implication glyph ⟹), **`supports` stays a single line**. Arrows
keep their existing direction: prerequisite → dependent.
---
## Why double-shaft (DECIDED 2026-09-14)
- **Semantically exact.** `follows` means prerequisite ⇒ successor — derivability. The implication glyph *is* the relation; nothing encodes "derivable from" more literally.
- **Weight where the structure is.** The DAG backbone (`follows`) gets the heavy glyph; the majority relation (`supports` is over half of all edges) stays light. The structural skeleton reads first.
- **Shape survives everything color can't.** Cluster regions already own the color channel; a shape code collides with nothing and survives colorblindness.
- **Scale-stable.** Pure geometry degrades nowhere — see the rejected alternative below.
## Why not dashed (rejected 2026-09-14)
Dash patterns break under our **constant screen-space stroke width**: the dash array is re-projected across zoom levels, so the pattern pops and crawls instead of scaling cleanly (the same class of artifact as the non-integer blur radii). And with `supports` the *majority* relation, dashes would dominate the canvas rather than mark the exception. Dashed = wrong twice.
## Geometry
The renderer is the custom 2D canvas; edges draw in the edge layer with shafts + arrowheads from `arrowGeometry` (sigma-style head, prerequisite → dependent). For `follows`:
- Both shafts offset along the edge normal by a **fixed screen-space gap**.
- **REVISED 2026-09-15 (owner rejected the first cut):** the converged variant — two tails merging into one shared arrowhead — read as two arrows feeding a single point, not as parallel strokes. The glyph is two **complete regular arrows** (own shaft + own head each) translated ±half-gap along the normal: parallel shafts AND parallel heads, nothing converges.
- The gap is proportional to the constant stroke width — a constant visual ratio at every zoom level.
- **No LOD collapse (withdrawn 2026-09-14).** The earlier idea of collapsing to a single shaft below a stroke-width threshold assumed width shrinks with zoom — it doesn't; stroke width is constant on screen, so the double shaft renders identically at every zoom level. Simpler than proposed: nothing to gate.
## Legend and scope
- Add a legend entry to the existing graph legend: `⟹ follows · → supports`.
- Main graph view only. The plan-block renderer (dagre) is untouched — the plan DAG is single-relation, so there is nothing to distinguish there.
## Lane
lh-graph's renderer files (`graph-canvas.ts`, `graph-view.tsx`). Not assigned, not scheduled.
docs/vision-extensibility.md
-98
View File
@@ -1,98 +0,0 @@
# The learning model & the extensibility direction
Owner-ruled 2026-10-02. Brief by design — the master mental model and the
extensibility philosophy in one place. Product detail lives in
[`vision.md`](vision.md); this is the frame over it.
## The three-part model
The product is one loop with three organs:
1. **The knowledge world model** — reasoning chains over propositions
(`chains.jsonl`, the stored truth). This is the *compressed* form of the
learner's understanding: what they hold, structured as it holds together.
2. **Learning sessions install it** — teaching sessions deposit
understanding into the model via extraction. But a session installs only
a small set of **weak cues**: the contexts in which the knowledge was
encountered. Installation alone is not recall.
3. **Practice strengthens recall** — for knowledge to connect at the right
moments, the **cue set** that triggers it must be broadened and
strengthened. That is practice's entire job. Practice outcomes feed back
into the model as **assessment levels** (per-proposition or per-chain —
TBD) — a measurable picture of what the learner actually holds — which
drive scheduling and the selection of learning activity.
```
install (sessions) → strengthen cues (practice) → assess (levels)
↑ │
└────────── schedule next activity ←───────────┘
```
Consequence: the world model and its levels are **core truth**; anything
that merely *strategizes* over them is a candidate for optionality.
## Roadmap — where we are (2026-10-02, owner-approved sequence)
Foundations complete: push architecture landed (zero standing polls),
per-session models, extraction pipeline (chains-primary), practice
problems v1, coverage UX, tool-result diet. The sequence from here:
1. **[IN PROGRESS] Quick-fixes round** — ask-questions must not collapse
in teach phase (quiz only); ESC aborts the turn from a pending
question behind a confirm popup (ESC on the popup cancels — no
double-abort); confidence-slider prompt rewritten as clean doctrine
(design reasoning must not leak into the system prompt). The old
practice dictation bug is deliberately UNFIXED (problem sets likely
abandoned).
2. **Adaptive practice** (the main arc) — session-shaped practice reusing
the session machinery + quiz tool; input = propositions/chains via the
generator dialogue (cluster/nodes); agent writes problems one after
another, adapting difficulty to hold learner accuracy ≈ 85%
(research-backed target); problem-engineering theory shared with v1;
no user-facing problem parameters (they live in the theory for the
agent; the learner steers by talking). Design round first: session
kind, spawn flow, what the agent receives, adaptation instructions,
end conditions, model slot.
3. **Session compaction** — manual trigger + automatic token threshold
(threshold configurable; auto toggleable in settings). Must preserve
extraction coverage (compact-covered-only or extract-before-compact —
design). Engineer the compaction instructions (what's critical to
preserve).
4. **Module extraction** — the extensibility Stage 1, AFTER adaptive
practice proves itself: convert both practice variants (batch sets vs.
adaptive sessions) as the real modules the registry/API is shaped
from; the loser then gets deleted or demoted. Old-module fate decided
empirically here.
5. **Fun, when wanted** — data view (GitHub-style activity squares,
counts, longest chain, accuracy/confidence split by teaching vs.
practice, Brier score — calibration error is also review-targeting
signal); standalone bare problem generator (cheap once modules exist).
## The extensibility direction (plugin-shaped)
Strong fundamentals, then optional modules — nothing speculative baked into
the default experience. The reference model is Obsidian's two-tier system:
**core plugins** (bundled, toggleable, same API as external ones) and
community plugins (loaded from disk). We adopt the architecture without the
third-party obligation:
- **Data core, strategies modules.** Ledgers (chains, levels, transcripts)
are core; learning strategies — practice variants, schedulers, drills,
enrichments — are modules. A module reads the world model + levels,
proposes learning activity, reports outcomes back as level signals.
- **The skeleton** (all plugin systems share it): manifest → deferred
activation → registration into extension points (tools, job kinds,
settings sections, routes/nav, events) → curated context API → unload.
Our DI ports, tool composer, jobs pipeline, settings sections, and events
bus already are those extension points, unnamed.
- **Staged path.** (0) each new feature's wiring concentrated in one
composition file; (1) a module registry with built-in toggleable modules —
convert a real feature (practice is the candidate) and let the conversion
write the API; (2) harden the context from real use, add per-module state;
(3) external loader from `data/plugins/` — optional, maybe never needed.
- **Why:** learning strategies are learner-, subject-, and
situation-dependent — the axis along which features must be optional and
toggleable; and a stable module API makes each new experiment cheap to
build, cheap to drop.
docs/vision-extraction-coverage.md
-52
View File
@@ -1,52 +0,0 @@
# Vision — Extraction coverage: message-granular ledger, token-batched runs
Ideation settled 2026-09-15. Not yet planned, not yet assigned to a wave. This document is the decision record; `vision.md` § "The graph write channel" points here.
One line: the extraction pipeline stops re-reading whole sessions and starts reading **only what it hasn't covered**. Coverage is tracked per **message** — a covered-fingerprint ledger per chat — runs are batched under a **token budget**, and the uncovered remainder is *derived*, never separately tracked. Abort or error loses at most one in-flight batch; nothing is ever silently left unextracted, and a manual "extract the rest" affordance falls out for free.
---
## The problem (found 2026-09-15, live review)
The pipeline's emission checkpoint (`SessionCheckpoint {chatId, foldedAt}`) is **whole-chat granularity**. Every run rebuilds the *entire* transcript (`transcriptView` from message 0) for both the extractor and the auditor — so total extraction cost over a session grows quadratically-ish with its length, the auditor re-sees everything, and the checkpoint only answers "has this chat ever folded".
Worse, the trigger is `EXTRACTION.everyNQuizAnswers` — a count of answered quizzes not yet in the event ledger. Consequences:
1. **Quiz-less sessions never extract.** A pure discussion session (no answered quizzes) accumulates unextracted knowledge forever. Quiz density is the wrong signal for "is there something to fold".
2. **Coverage holes are invisible.** An aborted or errored run checkpoints nothing (correct discipline) — but the hole it leaves is only detectable by re-counting quiz answers. Content the pipeline hasn't covered is nowhere recorded. Sessions are not granular enough; the unit was wrong.
## The decision set (DECIDED 2026-09-15)
**The message is the unit of coverage.** Not the session, not a chunk of text. A coverage ledger per chat records **which messages have been extracted** — append-only, each entry `{fingerprint, tokens, foldedAt}`.
- *Why not infer coverage procedurally (from provenance):* provenance is a **lower bound** — a message the extractor saw and found nothing in leaves no trace, so "seen and empty" is indistinguishable from "never seen". An explicit record is required.
- *Why not chunk-identity:* if chunks are the recorded unit, one quiz answer merging into a message inside chunk 2 shifts token counts and can cascade boundary changes across every later chunk — mass re-opening. Message fingerprints localize the re-open to exactly the mutated message.
**Coverage = the covered set; "uncovered" is derived, never tracked.** No dual bookkeeping: at read time, fingerprint the current transcript's messages, diff against the covered set. The complement *is* the coverage view — and the input to any "run extraction on what's missing" affordance. Two ledgers would drift; one ledger plus a deterministic diff cannot.
**The fingerprint is over the extraction-relevant content**: role + text parts + tool parts *including their outputs* — exactly what `transcriptView` sees. It is stable under cosmetic mutation (plan-block part attach — data parts are invisible to the view) and changes exactly when the view changes.
- *The quiz-merge wrinkle, handled by the fingerprint:* when the learner answers, the SDK part-merge fills the output **inside the assistant message that asked** — a message the previous run already covered. The fingerprint changes, the message re-opens itself (and nothing else), and the next run re-extracts that one message. The same behavior today's full re-read achieved, bounded to one message.
**Tool calls and results are never separated (owner rule).** They live in the same message — coverage and batching are message-granular, and runs are consecutive-message groups — so a call and its result land in the same run by construction. No splitting rule is ever needed.
**Quiz events are not special.** No separate quiz-outcome channel, no deterministic quiz-event pass — quiz parts are messages like any other tool content. They ride re-opened messages through the normal pipeline; the merger's attribution and the existing ref-idempotent appends do the rest. Nothing in the pipeline treats them differently.
**Trigger: uncovered tokens ≥ the chunk budget.** Replaces `everyNQuizAnswers`. `EXTRACTION.chunkTokens` (a config knob; sizing at build time — the budget is really the transcript share of the extractor's context window, which also carries the graph and instructions; candidate range 2k–8k). Token estimate chars/4, same as the rest of the app. This fixes the quiz-less-session gap for free: text alone fires the trigger. Still fired at `onFinish` — content is settled there.
**Runs are token-budgeted batches of the oldest uncovered messages.** Group consecutive uncovered messages (by transcript position) into runs under the chunk budget; execute sequentially; **checkpoint coverage incrementally per merged run** — the today-discipline ("nothing merged, nothing checkpointed") becomes per-batch atomic. An abort or mid-run error loses only the in-flight batch; every earlier batch's coverage is already durable, and the remainder stays visibly uncovered. Single-flight per chat unchanged.
**Cost shape.** Chunking bounds the *transcript* share of each run; the auditor still sees the whole graph (identity needs it) — that share is untouched by this design.
## What it buys
- Abort/error never silently loses knowledge: the uncovered set is the standing record of what the pipeline hasn't covered, and it's derivable at any moment, per session.
- Long sessions extract linearly, not quadratically.
- The manual path is trivial: "extract uncovered content" = batch the diff and run. The UX (coverage readouts per session, an extract-now affordance, maybe a graph-view coverage strip) is a later wave — the data model makes all of it cheap.
## Parked
- The manual-run UI and coverage readouts (deliberately — the user experience gets its own pass).
- The one-shot context injection's cousin case: an aborted/errored injection is never retried for that session (one-shot ever). Not knowledge loss — a graph read — but if a manual re-run affordance lands for extraction, the injection could share it.
- The notes-import path (`src_` sources): whole-file semantics stay; files are one-shot imports, not growing sessions.
- `SessionCheckpoint`/`sessions.jsonl` migration shape: covered-set lines replace/augment the current whole-chat checkpoint; details at build time.
docs/vision-persistent-agent.md
-28
View File
@@ -1,28 +0,0 @@
# Vision — The Persistent Agent (PARKED)
Started 2026-09-27, ideation round. **Parked by the owner 2026-09-27** — "a much later thing." This document preserves the ideation so nothing is re-derived. Practice module and side chats go first; branching after those.
## The pitch
A **tutor of record** — an agent that teaches nothing. It is the general interface to the accumulated learner model: it knows where the session records live, can read them, holds access to the knowledge graph, and keeps a detailed, longitudinal record of the user's understanding.
## The substrate — sessions as markdown
- Sessions get converted to **markdown records, persisted as files** — the corpus other agents read with the read/ls/find/grep tools every pi agent already has. This is the pull-based (pi-native) version of context sharing, philosophically opposite to the removed Wave-7 pushed-injection.
- **JSON stays canonical; markdown is a derived export with a defined shape** — regenerable, no second source of truth.
- Two audiences, one artifact: the human archive and the agent food want the same *distilled session record* — goal, the plan map (already mermaid — markdown-native), nodes covered, checks with outcomes, misconceptions found, where the session left off (~40 lines, not a 400-line verbatim transcript; the verbatim JSON exists forever).
- Open: who writes the record — dumb code folding the record, or an agent summarizing at session end. Could ship standalone before the rest of this document (it serves export value immediately and this agent later).
## The agent itself (ideation, unruled)
- **Queries the graph, doesn't hold it.** The graph lives in stores with query tools; holding it as context text is stale-copy risk and token burn. Context stays small; truth stays in the stores.
- **user.md — a living synthesis the agent maintains** — with the discipline that **the graph is the truth and user.md is the index**: the graph is evidence-backed (every node traceable to extraction); free-form prose memory has no audit trail. Disagreements resolve toward the graph. Otherwise it's a second brain with no epistemics.
- **The graph-view composer becomes its interface** — the natural home: the page where you see everything you know, and ask about it. This also makes the graph a control surface: **sessions created from the graph** (select nodes → "teach me this" → a session whose plan could seed from the selection; the probe could focus or skip accordingly).
## Session kinds (relevant once practice exists too)
Three fundamentally different session types — teaching, practice, the persistent agent's chats — should not sit mixed in one flat list. Agreed direction: **one session store with a `kind` discriminator, three surfaces** — teaching in the sidebar (untouched), practice entering from the graph/sessions, the tutor-of-record chat at the graph composer. Icons alone say what a thing is; surfaces put it where its activity lives. (The store-side kind field becomes real as soon as the practice module lands — see vision-practice.md.)
## Home page
If practice and the persistent agent both enter through the graph, the graph stays home and grows denser with affordances — no tabs. Tabs are what you add when something has no natural home. Watch: the graph page must not accrete too many meanings; "everything I know, and everything I can do with what I know" is one coherent meaning, and it should stay that way.
docs/vision-practice.md
-44
View File
@@ -1,44 +0,0 @@
# Vision — The Practice Module (Exams)
Started 2026-09-27, ideation round. **Priority 1** of the post-explain roadmap (above side chats; persistent agent parked separately).
The teaching sessions are the *installation mechanism* — they install knowledge. Practice is the *training mechanism* — it solidifies what was installed. Sessions are for learning new things; practice is for retaining and sharpening them. Spaced repetition is the explicit long-term goal, but the entry point is deliberately simple: **the learner picks the nodes** (most commonly: everything a session covered) and gets an exam over them.
## The assessment record model (owner rulings 2026-09-27)
- **Session quiz answers NEVER carry over to the graph.** Quizzes in a teaching session tag *plan steps* — entirely session-scoped objects, distinct from global graph nodes. No direct mapping. (The graph only ever hears the extractor's judgment.)
- **The extractor's understanding observation is data point zero.** First assignment at node conception; each session's extraction adds its own observation (today's seam already emits these as session-scoped ledger events).
- **Each practice session that touches a node appends one new assessment record.** A node's understanding is therefore a *history of records* — never an overwritten field. "Understanding is inferred, never encoded" (vision.md §1) stays intact: the displayed level is derived from the record set.
- **Aggregation over the records is future theory.** Weighted average at minimum; a Bayesian flavor is attractive (few data points = low confidence; the number of records should modulate certainty). Not decided. The user is presented **a number**.
- **Labels are the extractor's standardization device; numbers are the truth.** The written scale (today: `solid / shaky / accepted / resisted / not yet engaged`) exists so extractor runs share one vocabulary; underneath it is an ordinal scale, and everything downstream — storage, derivation, presentation — works in numbers.
The record history doubles as the substrate for **revision suggestions** ("what should I practice next") — schedulable mathematically/algorithmically later. But suggestions are a derived surface; the primary entry is always the learner's own node selection.
## The exam flow (minimal agent shape, owner sketched 2026-09-27)
1. **Entry.** The learner chooses a set of nodes — most commonly "the propositions this session covered" as a shortcut over a session. (Coverage data already derives session → node sets.)
2. **The problem-engineer agent.** System prompt carries the problem-engineering theory and instructions (owner will author this theory — pending, see below). Input: the propositions and their connections — a portion of the graph, NOT session transcripts. Output: a problem set.
3. **Coverage is guaranteed and self-verifiable.** Every proposition in the input is tested by some problem, and **each problem's metadata encodes the propositions it tests** — so the agent itself can check completeness and omit nothing (the metadata reinforces its own verification).
4. **Exam shape.** Short and satisfying: **3–6 really good problems.** Challenging where possible — "all knowledge can usually be applied," so application problems are on the table across domains, not just math-exam shapes. (Owner leaning; theory pending.)
5. **Problems may be one long tool call.** The problem set is authored up front — solutions and answers written beforehand (commit-before-pose, the same sycophancy mitigation as `explain`).
6. **The learner turns in answers.** The agent's next input is the submission.
7. **Grading against its own pre-written solutions.** Then the grades are submitted **via a dedicated tool** — that tool's input is what updates the nodes' understanding records. The grading tool call is the write seam.
## Problem formats
- The two existing question shapes (multiple-choice `quiz`, free-recall `explain`) are solid problem shapes — but the practice module gets **its own purpose-fitted tools** (duplicates either way), so teaching tools stay untouched.
- **Formats stay highly configurable.** More shapes will likely be added; nothing is hardcoded to two.
- **The problem-engineering theory is an owner deliverable.** The philosophy of constructing the optimal practice problem (types, difficulty, application-vs-recall, how connections enter problems) — owner is thinking and will bring it back. No implementation of the generator until that theory lands.
## What this module finally unlocks
The live, continuously updating view of the learner's understanding: extractor observations as the floor, practice outcomes accruing on top, derived numbers on the graph. This is the node-understanding-assessment thread resuming.
## Open questions (unruled)
- Practice sessions as a third session kind in the store (`kind: practice`)? Where do exam records live, and do abandoned/partial exams persist?
- UI surface for practice sessions (exam presentation is meant to be satisfying — the shape of that screen is open; entry likely from sessions and/or the graph).
- Retake policy — does re-practicing the same exam append new records like any other practice?
- Migration: existing nodes already carry extractor observations — these become the initial records.
- Does the grading write path go through the ledger (as a new practice-scoped event kind) or a dedicated assessment-record store?
- Scheduling/revision-suggestion algorithm — parked until the history exists.
docs/vision-side-chat.md
-27
View File
@@ -1,27 +0,0 @@
# Vision — Side Chats (block-scoped clarification threads)
Started 2026-09-27, ideation round. **Priority 2** of the post-explain roadmap (practice module first — this one is expected to be easy at the end of the day).
Quick clarification for quick questions — *between* the teaching. A separate, minimal agent exists beside the main learning agents; the teaching agents must stay unloaded (the owner's ruling: side chats have **zero effect on the teaching session**).
## The shape (owner sketched 2026-09-27)
- **Per teaching block.** Each block of the session transcript (the units divided by separators — turn groups) can carry its own mini thread.
- **UI discipline — invisible until relevant:**
- No side chats → **nothing changes in the UI**, except:
- a small chat button in the **bottom-right corner of each block, visible on hover** of that block;
- opened threads render as a **collapsible mini-chat underneath their block**, accumulating inline.
- **Never part of the teaching record.** The teaching agent never sees these threads; they are annotations on the session, not conversation within it. Where they persist (same file under a teacher-invisible namespace, or a side store keyed by chat + block anchor) is open.
## The agent (v1, first try — owner ruled the starting point)
- **Block-only context.** The ONE block being asked about is its only input, plus a system prompt instructing it to answer **very non-verbosely, directly, simply**. Questions are about specific sections, and the UI makes that scoping clear.
- **Forking the session was considered and set aside.** A fork (session-so-far + different system prompt) gets role-confused — it reads the transcript as its own words. If block-only proves ungrounded, the agreed escalation ladder, in order: + the plan tree, + the relevant graph propositions, + selected earlier blocks.
- Bare, cheap, tool-less, one model slot down from the teacher. The honest risk on record: an ungrounded answer that confidently contradicts the lesson is poison under the accuracy doctrine — watch it, don't pre-solve it.
## Open questions (unruled)
- Block identity/anchor stability (blocks are derived groupings — anchor by the block's first message id).
- Persistence location and shape (teacher-invisible namespace in the chat record vs. a separate store).
- Model slot (cheap, presumably) and any context ceiling.
- Keyboard/affordance details (hover-only button, or also focus-reachable).
electron/main.cjs
+8
View File
@@ -36,6 +36,14 @@ const path = require("node:path");
const SIDEBAR_AUTOCLOSE = 928;
const PRODUCT_NAME = "LearnHarness";
// APP NAME → userData (owner, 2026-10-10): the Application Support dir is
// exactly "LearnHarness" in dev AND packaged. Packaged Electron would
// otherwise derive it from package.json "name" (now lowercase
// "learnharness"), and productName metadata alone doesn't reach app.name.
// Must run before the LH_DEV_URL suffix below — dev becomes
// "LearnHarness-dev".
app.setName(PRODUCT_NAME);
const AUTH_FILE = () => path.join(app.getPath("userData"), "auth.json");
const DATA_DIR = () => path.join(app.getPath("userData"), "data");
package.json
+2 -1
View File
@@ -1,6 +1,7 @@
{
"name": "learning-harness",
"name": "learnharness",
"version": "0.2.0",
"license": "AGPL-3.0-only",
"description": "A standalone learning harness: a teacher agent (AI SDK) with quiz/ask interactions, session artifacts, and a derived knowledge graph of what the learner knows.",
"type": "module",
"private": true,